Monitoring MySQL Import Progress

When importing a large MySQL dump file, use pv to monitor the progress of data through a pipe:
pv -petr dumpfile.sql | mysql -uroot my_database

This will give you a progress indicator such as:
0:00:05 [3.63MB/s] [===>        ] 0% ETA 0:09:41

mysql
osx

Comments (0)

Permalink

MSSQL Bulk Insert from CSV file

http://www.thescripts.com/forum/thread158756.html

BULK INSERT Test FROM 'X:\temp\Test.CSV'
WITH (
DATAFILETYPE = 'char',
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)

Could not bulk insert because file ‘X:\temp\Test.CSV’ could not be opened. Operating system error code 3(The system cannot find the path specified.).

http://support.microsoft.com/kb/316371

objBL.transaction = "FALSE"

mssql

Comments (0)

Permalink

IllegalStateException: getAttribute: Session already invalidated

http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/org/apache/catalina/cluster/session/DeltaSession.html

http://www.mail-archive.com/tomcat-user@jakarta.apache.org/msg124558.html

apache
tomcat

Comments (0)

Permalink

uninitialized constant MysqlCompat::MysqlRes

After getting the dreaded uninitialized constant MysqlCompat::MysqlRes error on OS X Snow Leopard, I finally found a solution.

Error:

$ rake db:migrate
...
rake aborted!
uninitialized constant MysqlCompat::MysqlRes

Solution from http://techliberty.blogspot.com/:

sudo gem uninstall mysql
sudo env ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

mysql
osx
rails

Comments (0)

Permalink

Installing/Upgrading Passenger + Nginx

After numerous searches, I found this post extremely helpful:
http://www.jotlab.com/2009/11/08/installing-ruby-on-rails-mysql-imagemagick-nginx-passenger-with-macports/

Highlights:

Install or update Passenger:
sudo gem install passenger

Find Passenger install path:
passenger-config --root

Configure MacPorts to use Passenger varient of Nginx:
cd /opt/local/var/macports/sources/rsync.macports.org/release/ports/www/nginx

Open config file:
sudo vi Portfile

Insert at the bottom of the file:

variant passenger description {Adds passenger support} {
    configure.args-append  --add-module=/Library/Ruby/Gems/1.8/gems/passenger-3.0.1/ext/nginx
}

Install/update Nginx:
sudo port install nginx +gzip_static +passenger +ssl

rails

Comments (0)

Permalink

SSH Tunnel in OS X

Open tunnel:

$ ssh -L 8080:remote-host:80 -f -C -q -N me@myserver.com

Close tunnel:

$ ps -aux | grep ssh
$ kill {pid}

osx

Comments (0)

Permalink

Quartz + Spring + Hibernate with JDBC Job Store

Here is how I configured Quartz to work with Spring and Hibernate. I’m using the Quartz JDBC Job Store which stores and retrieves the job settings from a database instead of a static configuration.

I use the SchedulerFactoryBean to manager the Quartz Scheduler lifecycle.

spring-servlet.xml

...
	<!-- Quartz -->
	<bean id="scheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
		<property name="applicationContextSchedulerContextKey" value="applicationContext"/>
		<property name="configLocation" value="classpath:quartz.properties"/>
	</bean>
...

The Quartz settings are stored in a quartz.properties file so that they can be unique to each instance of the web application. This setup requires that quartz.properties be on the classpath. This file defines the JDBC connection for Quartz to use.

quartz.properties

...
org.quartz.dataSource.myDS.jndiURL=java:comp/env/jdbc/MyDS
org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.dataSource=myDS
org.quartz.jobStore.tablePrefix=JB_
org.quartz.jobStore.useProperties=true

MyBaseJob

public abstract class MyBaseJob extends QuartzJobBean
{
    private transient ApplicationContext applicationContext = null;

    public ApplicationContext getApplicationContext()
    {
        return applicationContext;
    }

    public void setApplicationContext(ApplicationContext applicationContext)
    {
        this.applicationContext = applicationContext;
    }
}

Each of my job classes extend MyBaseJob, overriding executeInternal. The jobs can use the applicationContext to lookup Spring service beans. In my environment these service beans extend HibernateDaoSupport.

    protected void executeInternal(JobExecutionContext ctx) throws JobExecutionException
    {
        MyServiceManager sm = applicationContext.getBean("myServiceManager"); // lookup Spring service bean
        ...
    }

hibernate
java
quartz
spring

Comments (0)

Permalink

Find and Replace String in Multiple Files

Periodically I need to replace a string in multiple files on Linux. I find myself looking up this command or a variant every time.

grep -rl OLDSTRING *.FILEEXTENSION | xargs perl -pi~ -e 's/OLDSTRING/NEWSTRING/'

linux

Comments (0)

Permalink

SSL + gzip Compression with Apache and Tomcat

My setup is a single Fedora Core 6 server running Apache 2.2.4 (with mod_ssl and mod_proxy) and Tomcat 5.

With mod_ssl the following virtualhost is setup using an SSL certificate from GoDaddy. Notice the SSLCertificateChainFile /etc/httpd/conf/ssl.crt/gd_intermediate_bundle.crt reference. This intermediate certificate comes from GoDaddy and is required. I didn’t catch this at first and couldn’t understand what my web browser was unhappy about. When I called GoDaddy support they told me my certificate was setup correctly and worked in all of there “off-site” test browsers. They were no help, so I continued to dig around and finally found the answer.

The mod_proxy lines pass all requests to the Tomcat instance listening on port 9014.

/etc/httpd/conf.d/ssl.conf

<virtualhost>
DocumentRoot  /var/www/html/
ServerName    myserver.com:433  SSLEngine on
SSLCertificateFile /etc/httpd/conf/ssl.crt/myserver.crt
SSLCertificateKeyFile /etc/httpd/conf/ssl.key/myserver.key
SSLCertificateChainFile /etc/httpd/conf/ssl.crt/gd_intermediate_bundle.crt
SetEnvIf User-Agent ".*MSIE.*"
         nokeepalive ssl-unclean-shutdown
         downgrade-1.0 force-response-1.0
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
CustomLog logs/ssl_request_log
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

ProxyPreserveHost On
ProxyPass / http://localhost:9014/
ProxyPassReverse / http://localhost:9014/
SetEnv proxy-nokeepalive 1
</virtualserver>

For the Tomcat setup, a proxy connector is configured to listen on port 9014 and to proxy port 443 requests (SSL). The communication between Apache and Tomcat is not secure, but this is not a concern since this communication is local to the server. Finally, compression is turned on for several common mime-types.

server.xml

  ...
  <Connector acceptCount="100" connectionTimeout="60000" disableUploadTimeout="true" port="9014" redirectPort="8944"
  scheme="https" proxyPort="443" compression="on" compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript"/>
  ...

apache
linux
tomcat

Comments (0)

Permalink

Problems using Digester in custom Struts Plugin

In migrating from Tomcat 5.0 to Tomcat 5.5, one of my custom Struts plugins started failing on startup with the following exception:

org.apache.commons.digester.Digester startElement
SEVERE: Begin event threw exception
java.lang.ClassNotFoundException: com.mikwat.struts.config.AuthorizationActionConfig

The root of the problems appears to be that the Digester defaults to using the same ClassLoader that loaded it and whichever ClassLoader this is, it doesn’t have access to my webapp class com.mikwat.struts.config.AuthorizationActionConfig.

After Googling around, I eventually came across this post regarding a similar problem loading the Quartz plugin: Mailing list archives.

The solution is to call setUseContextClassLoader(true) on the Digester object which forces it to use the ClassLoader found by calling Thread.currentThread().getContextClassLoader().

apache
java
tomcat

Comments (0)

Permalink