Skip to content
January 7, 2005 / TildeWill

1337 Thread Programming Master

Every morning I have a set of spreadsheets scheduled to run. Each of these spreadsheets has a lits of reports to grab from Oracle. It grabs these reports by sending a string to the command line to be executed. There are about 50 of these reports scattered across a dozen spreadsheets. More or less all of them run in the morning before anyone gets here (long before I ever want to wake up). Part of the reason they were in so many files is that one machine can’t run ALL the reports at once and the way the old spreadsheets worked, they just threw out the strings to the command line until it spit them all out or the computer crashed (if it was more than 10-12 the later was most always the case).

Now there was a field to specify “number of threads” but it wasn’t true threading in that it would wait for all “threads” to finish before starting a new thread. So say you have a total of 5 reports, you specify 4 threads and reports 1-3 take 5 minutes to run, but thread 4 takes 1 hour. Well instead of report 5 starting after 5 minutes it waits until that long ass report is done. Retarded, I know.

Here’s the catch, it is tough to tell when a report is done running since an external program takes care of that and doesn’t really return anything useful. It does however generate files for the reports. Great, just check when the files are created. Wrong! The files are created then appended incrementally, you’ll never know if they’re done unless you are able to open them. Ok, so wait until you can open the file. Sounds good, but if no data is returned by a report, no file is ever generated. Whoops! So now you’re forced to look at the log file and see if it was modified AFTER the report running process started. WHEW! So now we know when the thread should be done.

Add a million loops and Sleep calls and badda boom badda bing it is a wonderful thing, real threads. SO now I can dump all the reports into one spreadsheet to run in the morning and minimize my hunting down which spreadsheet that was in… okay… what machine is that on… okay… this one ran half way it seems…. yeah it sucks. Now, less suck.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: