Java DB connections in the real world

Yes I’m muttering about relational database connections AGAIN (prepared to be bored to within an inch of your life), and why you ask, because as a domino person who now makes most of his money from none domino projects, as is my current team leader, we are used to database connections being something you just ask for and then forget, they are always there, they don’t time out and they handle them selves quite beautifully, not like the java lot, for all you “domino is old and naff” lot, i’ll take you on with database connection any day of the bloody week, anyway enough ranting.

Technologies used/supported:

Jboss 4.0.5,
MYSQL 5,
hibernate 3.2
C3p0 1.7.2
Java 1.6

These kind of issues (including the Quartz based one I’m doing an article on later), all come about because we are trying to do things properly, and by properly i mean the same .War file works the same all the way from dev to Live, because it gets its connection info from con-fig documents on the server/db rather than hard coded into the app

Soooo

Now your all used to seeing a persistence.xml or hibernate.cfg.xml file that has stuff in it like this

<property name=”dialect” value=”org.hibernate.dialect.MySQLDialect” />
<property name=”hibernate.connection.driver_class” value=”com.mysql.jdbc.Driver” />
<property name=”hibernate.connection.url” value=”jdbc:mysql://127.0.0.1:3306/SETTINGS” />
<property name=”hibernate.connection.username” value=”root” />
<property name=”hibernate.connection.password” value=”rootpassword” />

(if you not, then tell me in the comments and Ill do an “from basics” blog)

All well and good and fine for dev and such, but not what you want in your actual distribution, what you want is something that just says “can i have the ‘setting db’ please”, and for that you need a ‘data source’, so swap out all the database specific lines for the following

<jta-data-source>java:/MYDATASOURCE-DS</jta-data-source>

Then create an xml file (mines called “data-source.xml” in your deployments directory, in my servers case “d:jboss4.0.5serverdefaultdeploy”, and put the following in, filling in the correct gaps

<?xml version=”1.0″ encoding=”UTF-8″?>
<datasources>
<local-tx-datasource>
<jndi-name>MYDATASOURCE-DS</jndi-name>
<connection-url>jdbc:mysql://127.0.0.1:3306/SETTINGS</connection-url>
<driver-class>com.mysql.jdbc.Driver</driver-class>
<user-name>root</user-name>
<password>rootpassword</password>
<exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter</exception-sorter-class-name>
<metadata>
<type-mapping>mySQL</type-mapping>
</metadata>
</local-tx-datasource>
</datasources>

You will need to put the appropriate driver *.jar file in the lib directory of your server (rather than in your app deployment .war file), in my case “d:jboss4.0.5serverdefaultlib mysql-connector-java-5.1.7-bin.jar”, restart your server and ta-da!, you suddenly have a server level data source.

Now in a sane world this should be all you need, but “i don’t think we are in Domino any more toto”, we are in the land of java, you will get lots of weird things happening from time to time but if you are using MYSQL the first one you will encounter is MYSQLs default timeout period of 8 hours, i.e. if you don’t do something every 8 hours the server will chuck you off and refuse to let you back on, unless you redo your whole connection, which in the case of a data source means a server reboot, ARSE!, one solution is to extent the timeout to the max you could want, but that stinks as a solution, a better solutions is to introduce connection pools, which means we introduce yet another layer of abstraction, this one designed to maintain control of the load a application puts on its server and to ensure those connections are available to the application, we will be using c3p0, not only is it the most popular one but its also has the daftest name (we are a star trek house).

First thing is to download c3p0 (it does come with hibernate but for purposes of instructions we will do it from scratch) from http://sourceforge.net/projects/c3p0/ and copy c3p0-x.x.x.x.jar to the same lib directory as you put your database driver, next you need to REPLACE the existing data source file with a new xml file (my new one is called “data-service.xml”) containing the below:

<?xml version=”1.0″ encoding=”UTF-8″?>
<server>
<mbean code=”com.mchange.v2.c3p0.jboss.C3P0PooledDataSource”
name=”jboss.jca:service=DataSourceBinding,name=MYSQL-DS”>
<attribute name=”JndiName”>java:MYDATASOURCE-DS</attribute>
<attribute name=”JdbcUrl”>
jdbc:mysql://127.0.0.1:3306/SETTINGS
</attribute>
<attribute name=”DriverClass”>com.mysql.jdbc.Driver</attribute>
<attribute name=”User”>root</attribute>
<attribute name=”Password”>rootpassword</attribute>
<!– <attribute name=”AcquireIncrement”>3</attribute> –>
<!– <attribute name=”AcquireRetryAttempts”>30</attribute> –>
<!– <attribute name=”AcquireRetryDelay”>1000</attribute> –>
<!– <attribute name=”AutoCommitOnClose”>false</attribute> –>
<!– <attribute name=”AutomaticTestTable”></attribute> –>
<!– <attribute name=”BreakAfterAcquireFailure”>false</attribute> –>
<!– <attribute name=”CheckoutTimeout”>0</attribute> –>
<!– <attribute name=”ConnectionTesterClassName”>0</attribute> –>
<!– <attribute name=”Description”>A pooled c3p0 DataSource</attribute> –>
<!– <attribute name=”FactoryClassLocation”></attribute> –>
<!– <attribute name=”ForceIgnoreUnresolvedTransactions”>true</attribute> –>
<attribute name=”IdleConnectionTestPeriod”>300</attribute>
<!– <attribute name=”InitialPoolSize”>3</attribute> –>
<attribute name=”MaxIdleTime”>600</attribute>
<!– <attribute name=”MaxPoolSize”>15</attribute> –>
<!– <attribute name=”MaxStatements”>0</attribute> –>
<!– <attribute name=”MaxStatementsPerConnection”>0</attribute> –>
<!– <attribute name=”MinPoolSize”>0</attribute> –>
<!– <attribute name=”NumHelperThreads”>3</attribute> –>
<!– <attribute name=”PreferredTestQuery”></attribute> –>
<!– <attribute name=”TestConnectionOnCheckin”>false</attribute> –>
<!– <attribute name=”TestConnectionOnCheckout”>false</attribute> –>
<!– <attribute name=”UsesTraditionalReflectiveProxies”>false</attribute> –>
<depends>jboss:service=Naming</depends>
</mbean>
</server>

The reason you replace the xml file, is that you will most likely be using the same data source name, and they have to be unique to the server, as you can see i have put all of the possible attribute names in the example as they are not documented very well and it was a bugger to find them, but commented them out, you should only really need “MaxIdleTime” and “IdleConnectionTestPeriod”, make sure its set to less than 8 hours (or what ever your db server timeout is), now things should just work after a restart, but you will get a bit more ‘static’ on your console, as c3p0 chatters a bit to the console,

there you go another riveting read, yell if bits are unclear

The Writing is on the Wall (Training Pt 1)

On with my preparations for the suspected tapering off of my role at my current main client, first part…TRAINING

Training is one of the things that all techs like to do differently, personally i like a combination of videos i can have on repeat in the background and building practical apps to solve problems. but their are some core bits that i recon that everyone on a training frenzy should have

1) vmware workstation (or virtual box or parallels or what ever rocks your boat), if your not using virtual machines, then start!!, with them you can mess about and try things, then bin them when they go wrong, top tip: build a ‘core machine’ and clone it each time you want to play with something new, that way you don’t waste time building up the base OS each time you want to try something.

2) http://www.safaribooksonline.com , the cheapest and best way to get at all the technical books you could want for one price, they also have a really good line in videos included in the price, yes i know all about the practical side being the important, but as i have discovered with flex and spring, that just because you are making it work does not mean you are doing it right, so OFFICIALY RATIFIED base theory is needed as well (especially for interviews), understandably safari had “stuff all” new Domino training material, but all is not lost (more on that later),

now on the to individual things i have to learn

Flex 4 and Adobe Air

Thankfully this is as easy as ever with adobe really leading the way when it comes to documentation, all varieties of learning stuff can be found at http://flex.org/resources with my personal fav being the “flex in a week” video and work sheets (already updated for flex 4) found at http://www.adobe.com/devnet/flex/videotraining/

Xpages

Next up is xpages, now i hear there is a book coming form the powers at IBM, but alas i cant wait, and actually i don’t have to (ha ha!), because there is good resources out there from the community, the 2 important ones being:

http://xpages101.net/ (a paid for video training site produced by Matt White who co wrote IBMs internal xpages training course,and has presented on xpages at every ‘lug’ and notes conference you can mention, the content is excellent though as i know Matt as a mate I’m always the expecting him to say at the end of each lesson “so that’s done, shall we go for a pint”),

http://xpagesblog.com/ (a community technical blog that is stuffed full of juicy xpages bits and an excellent read)

There is also a spectacular 40+ series of xpages articles that i have lost the link to (will dig up and update here)

Granite Data services

This is just a “just build the darn thing and make it work” job but to my relief the guys and gals at http://www.graniteds.org have written very good documentation thank you, found at http://www.graniteds.org/confluence/display/DOC/Documentation+-+2.1

Jquery

God damn where to start, hundreds of good sources, lots of them free, but of them all i ended up with the purchased book “JQUERY: NOVICE TO NINJA” found at http://www.sitepoint.com/books/jquery1/ then worked my way thought the some usefull looking jquery pluggins taking them apart ( http://www.gmarwaha.com/jquery/jcarousellite/ and http://bassistance.de/jquery-plugins/jquery-plugin-validation/ )

Spring

This is such a hideous subject, that you just have go to http://www.safaribooksonline.com and look thought all the myriad spring books they have until you find one you like the style off, then sit down with a pack of head ache tablets and grind your way through it, you will swear a lot, then suffer an moment of Epiphany after which its just a case of filling in the mad java developer “lets make this elegant” gaps.

Adobe illustrator

Strangely Adobe aren’t as good on this as they are on flex (maybe because they have everyone by the short an curlies on it as a market leader) so its back to http://www.safaribooksonline.com where there is a excellent video series and the normal good books, practice practice party people

Android dev

This is a bit like jquery in that there is simply tones of content, I’m going to use a mixture of http://developer.android.com and http://www.safaribooksonline.com content combined with practice apps ( although as i don’t have a android phone I’m using the a vmware image of the live android image http://code.google.com/p/live-android/, thanks to the wretch for pointing this one out)

Right, back to work!!

Fitness Week 4

2 weeks without a technical article just updates on the feeble efforts of the ‘Green’ and I trying to not be spherical is poor, very poor, but the stalemate holds, however the viciousness increased, with Mr Green giving me a savage kicking on Tuesday at squash and me returning the favour in spades today, Ha!!,

First person to win 250 squash games
Kieran: 7 Games
Mark: 19 Games
(1 point head start on a game for every 10 games the other person is ahead)

First person to loose 2 stone
Kieran ( Initially 15 Stone, 8 pounds) : 15′ 2″
Mark ( Initially 16 Stone, 8 pounds) : 16′ 7″ (still no change boo hoo!)

I think Mr Green has installed a tape worm for his weight loss!!

Fitness Week 3

Still a draw, i do however think Mr Green is being unfair by trying to eat properly and moving up to maybe 6 sessions a week

First person to win 250 squash games
Kieran: 3 Games
Mark: 15 Games
(1 point head start on a game for every 10 games the other person is ahead)

First person to loose 2 stone
Kieran ( Initially 15 Stone, 8 pounds) : 15 3″
Mark ( Initially 16 Stone, 8 pounds) : 16′ 7″ (sobs!!)

The Writing is on the wall (The initial conclusion)

As a contractor you know in the back of your mind that nothing can last, but humans tend to prefer a warm fluffy feeling and try to forget this fact, however my largest client has taken on a large outsourced company (its pretty much a run of the mill outsourcing, nothing that reflects badly on the company, I’m presuming its being done so that someone somewhere gets a tidier company org chart as in the long term it sure as hell is not going to save them money, i heard the phrase “strategic partnership” and thats never a good sign), unsurprisingly said company has moved to ‘acquisitive’ mode as soon as they had their foot in the door (to be fair they would be daft if they did not), this means that i suspect that time is limited (hence the countdown timer on the side of this page).

I however have more time (hopefully) to cater to this movement than lots of other people do in similar situations, and my CV is nearly as i would want it, but it could do with a bit more shinning, will therefore be trying to get the following down pat in the next few months, even if I’m wrong such a exercise wont do any harm at all.

Need to get a grip on
Granite Dataservices
Flex 4
get better at Xpages (compared to my other domino dev, i suck)
get better at Jquery (i use it but don’t UNDERSTAND it)
get better at Spring (i have practical knowledge but would suck at a formal interview)
Adobe illustrator (im ok in a “its C.A.D in colour” way, but want it for the flex 4 skinning and chibi images which means i need to be better)

Time Permiting
Android dev
Adobe Air

Its a lot of work but i have 160 days, now i just have to stop working for clients all the time so i have time to learn something (mixed blessing)

Client webservice timeout on CXF webservice on Jboss

What a riveting blog title, but this one is mainly for people to find for a solution via google.

Problem: you have a JBoss server with a webservice CLIENT talking to a webservice somewhere else, and you keep getting timeouts, most likely with the following error

org.apache.cxf.phase.PhaseInterceptorChain doIntercept 
INFO: Interceptor has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: Could not send Message.
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:64)
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:226)

If this happens after 60 second (the CXF default timeout), then it is most likely not the target server refusing you, but your client timing out (if your luckily you can get to the target server logs and confirm). Sooo lets extend the client timeout period, personally i like doing this via server configs and indeed that was my first port of call (do a google search for CXF.xml for more details), however even after i figured out what to call the file (cxf.xml, jbossws-cxf.xml, or jboss-cxf.xml) and got it in the right place (inside the WEB-INF of the project for access via POJO’s) the darn thing just ignored me, incidentally the following is the “meat” of the file which should change the timeout (in fact the “*.http-conduit” changes the timeouts on all webservice clients on that app.

<http-conf:conduit name="*.http-conduit"> 
<http-conf:client ConnectionTimeout="120000" /> 
</http-conf:conduit> 

After a prolonged swear, and discovery that i was not the only person who had had this problem on JBoss for CFX clients, I binned that and went into the code, now if your getting this problem then your all ready making a web service client call so you will already have a line like this

Service client = service.yourwebservice();

if you cant find one, and make all you “client stubs” with something like WSDL2Java or via your IDE, do a search for “extends Service”, you are now looking for a line like this

public class myfunkyservice_Service extends Service { 

when you find it, it will tell you what you should go back and look for ie

funkyservice = new myfunkyservice_Service(URL);
myfunkyservice_Service client = funkyservice.yourwebservice();

well something like that anyway, now bung the following code in after it (changing ‘myfunkyservice_Service’ to what ever you client service is called, and not forgetting that the timeout is in milliseconds)

Client cl = ClientProxy.getClient(myfunkyservice_Service);
HTTPConduit http = (HTTPConduit) cl.getConduit();
HTTPClientPolicy httpClientPolicy = new HTTPClientPolicy();
httpClientPolicy.setConnectionTimeout(120000);
httpClientPolicy.setReceiveTimeout(120000);
http.setClient(httpClientPolicy);

that’s the code done, it should all work swimmingly, one exception to this is if you get the error

java.lang.ClassCastException: com.sun.xml.ws.client.sei.SEIStub
at org.apache.cxf.frontend.ClientProxy.getClient(ClientProxy.java:93)

What this bundle of joy means is that sun’s library is loading with a higher priority that the CFX lib, if this is on a server its time to get dirty with your config, if this is on your IDE then, go to your projects properties –> Java build path –> Order and export, and bump the CXF lib up (try it at the top)

there you go, now wasn’t that fun 🙂
Old Comments
————
##### Nancy(15/11/2011 20:01:02 GMT)
Thank you very very VERY MUUUUUUUCHH!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Really
##### Mark Myers(16/11/2011 19:58:48 GMT)
@Nancy your welcome, glad it helped
##### Mark Myers(27/11/2011 21:48:18 GMT)
@Anneta no prob
##### lucky7175(20/12/2011 06:27:43 GMT)
It works this way

webClient = WebClient.create(URI);
HTTPConduit conduit = WebClient.getConfig(webClient).getHttpConduit();
conduit.getClient().setConnectionTimeout(120000);
##### lucky7175(20/12/2011 06:08:44 GMT)
I dont have control on server code and we use cxf webclient. Can u Pls. let me know how i can set only in client side
##### Mark Myers(22/12/2011 14:42:50 GMT)
yup thank you luckly, isn’t that snippet of code already in the blog at the bottom?

Making any ISO

While building a new win7 VM, I discovered that some of my software was still on DVD/CD only (the horror), I figured that there must be a easy linux way of making an iso image, and diddle me so there is, its called dd The ultimate frenzy guide to how to use this amazing command can be found at How To Do Everything With DD but for me it boiled down to

dd if=/dev/scd0 of=DVDBACKUP.iso

you just need to know where your dvd drive is mounted

$ mount

/dev/sda1 on / type ext3 (rw,noatime,errors=remount-ro)
/dev/sda3 on /home type ext3 (rw,noatime)
/dev/sda1 on /media/SecondHDD type ext3 (rw)
/dev/scd0 on /media/cdrom0 type udf (ro,nosuid,nodev,user=myers) <– This One looks Right

Sooo

myers@EX-L1:~$ sudo dd if=/dev/scd0 of=DVDBACKUP.iso

7816384+0 records in
7816384+0 records out
4001988608 bytes (4.0 GB) copied, 312.694 s, 12.8 MB/s

Cooooor, instant .iso or any order disk copy you might like (its built into all good OS’s, and by that i mean all with a Unix core { yes yes the turtle neck fruit boys can even be included on this one})

Fitness Week 1 AGAIN

Back at it again, this time I’m dragging a fellow member of LDC with me, Mr green and i are having a double competition, each with a prize (a slap up meal):

First person to win 250 squash games
Kieran: 0 Games
Mark: 3 Games

First person to loose 2 stone
Kieran ( Initially 15 Stone, 8 pounds) : 15 8″
Mark ( Initially 16 Stone, 8 pounds) : 16′ 8″

MAY THE BATTLE BEGIN!!!