Roy's Blog

I am proud to announce the release of JWatch v0.8 – An Open Source and Free Monitoring Application for the Quartz Job Scheduler. The browser-based user-interface allows developers to monitor multiple Quartz instances in real-time. In addition, a RESTful API is available for those wishing to extend the application.

Future releases will expand JWatch’s capabilities with the addition of management features, designed to make it easy for users to change Quartz Jobs and Triggers in real-time and monitor the changes.

Note: The current release of JWatch works with Quartz 2.0+. It will not work with previous versions of Quartz.

The current release of JWatch, includes the following features:

• Monitor multiple Quartz Instances across multiple applications concurrently.
• Monitor multiple Schedulers and Jobs per Quartz Instance.
• Drill-down for more information in to Quartz Job and Trigger objects.
• Real-Time monitoring of presently executing jobs.
• RESTful JSON API for those wishing to extend the application.
• Easy to deploy – just drop in a war and configure your instances.
• It’s free! (LGPL)
• Easy installation and configuration.

Screenshots:

Links:

• JWatch – A Quartz Monitor Home
• Installation / Configuration Documentation
• RESTful API Documentation

I’ve been working on an OSS project during my spare time which is deeply intertwined with Terracotta Quartz. Because of the nature of the project, I do not interface with Quartz through the standard API (no bundling), but am accessing it via JMX MBean. This is not an easy task, as there are large differences between Quartz 1.x and Quartz 2.x exposed methods and bundling Quartz in the project distro is not an option. Since I plan to support different versions of Quartz, it is left up to my adapters to figure out what version someone has installed and how it should access remote methods. Any guess to what I’m building?

One thing I was unable to find is how the important Quartz objects relate to each other. In the past, I’ve implemented Quartz in projects, but never really dug around at its internals. Even while implementing Quartz, I found I often ended up with a one-to-one relationship between the major components. Instance -> Schedule -> Group -> (many)Job.

But wouldn’t you know it – Quartz is a lot more flexible than that, allowing many-to-one relationships for several components. Here is a rather raw diagram of how all the pieces pay together. I didn’t do a very good job of showing the many-to-one relationships in the model, but you get the point.

Frankly, I would never have discovered this under normal use, and had to dig around with the JMX MBean to visualize this. It may be documented somewhere, but I couldn’t find it.

Quartz’ flexibility is great, but building a UI around this is going to be interesting.

The most recent Terracotta Quartz release seems to have changed some of the API methods to list scheduled jobs. Although minor changes, I thought I’d post sample code here for anyone trying the same thing as I, and hitting a wall with the API differences. Some of the changes are documented here in the latest release, but it’s OSS, so expect the docs to suck.

The changes, as best I can tell (correct me if I’m wrong), revolve around accessing the Scheduler object.You can see the differences between the 1.8 API and the 2.0 API.

Changes I found:

• scheduler.getJobGroupNames() now returns a List<String> , and not a String[].
• scheduler.getJobNames(groupName); has dissapeared.
• You must now use scheduler.getJobKeys(GroupMatcher.groupEquals(name)); to get a Set<JobKey>, which will be used to call scheduler.getJobDetail((JobKey)iter.next());

Sample Code

      try
      { 
         Scheduler scheduler = StdSchedulerFactory.getDefaultScheduler();
         List jobGroups = scheduler.getJobGroupNames();
         for (int i = 0; i < jobGroups.size(); i++)
         {
            String name = (String) jobGroups.get(i);
            Set keys = scheduler.getJobKeys(GroupMatcher.groupEquals(name));
            Iterator iter = keys.iterator();
            while (iter.hasNext())
            {
               JobDetail jobDetail = scheduler.getJobDetail((JobKey)iter.next());
               JobDataMap jobDataMap = jobDetail.getJobDataMap();
               // do something
            }
         }
      }
      catch (SchedulerException se)
      {
         se.printStackTrace();
      }

I specially like the addition of scheduler.getJobKeys(GroupMatcher.groupEquals(name));, where GroupMatcher can take some crude expressions like groupContains(String compareTo) and groupStartsWith(String compareTo).

A recent experiment with IMF, CIA, and World Bank datasets had me fighting to map ISO 3166 Alpha-3 codes to ISO 3166 Alpha-2 codes. The reason I had to perform the mappings, is because most downloaded flag icon libs on the web are based on Alpha-2 codes.

I have managed to map 183 countries in my local database, and am making the clean CSV available for download here.

The format is:

ISO ALPHA-2 Code , ISO ALPHA-3 Code
AF,AFG
AX,ALA
AL,ALB
DZ,DZA

As I work on a new site, I hit a bit of a wall when trying to calculate GDP Growth Rate, given Quarterly GDP figures (in Billions $). I had a very hard time trying to find this through the almighty Google, and a lot of the formulas on the web, are either wrong or simply cover how GDP is calculated, which is not what I needed:

Y = C + I + E + G, where
Y = GDP
C = Consumer Spending
I = Investment made by industry
E = Excess of Exports over Imports
G = Government Spending

So dusting off some old Economics books, the formula for calculating Annual GDP growth over two periods (usually measured in Quarters) is:

(((x(t)/x(t-1)) ^ (number_of_observations_per_year)) – 1) * 100

Where:

• ‘x(t)’ is the value of GDP x at time period t.
• ‘x(t-1)’ is the value of GDP x at time period t-1, or the previous Quarter.
• ‘^’ means, to the power of.
• ‘number_of_observations_per_year’ in this case would be 4.

So given values (in Billions $) for Q3 and Q4 2010, we have:

(((13370.10 / 13278.50)^ 4) -1 )* 100 = 2.8%

Now hopefully Google will lead the other 3 people in the world that are looking for this formula here.

I’ve had a lot of time on my hands lately. I’ve mostly spent it playing with fish tanks, but as of last week, I was all out of fish work and room for new tanks… so I started talking to a few Atlanta startups. I basically go in, they hammer me with questions and I do my best to not look stupid. After what I helped build at LoopFuse, you’d be hard pressed to find someone in this town that can build something that secure, scalable, fault-tolerant and damn intuitive to use! This isn’t San Francisco, and yes, that was an arrogant statement.

There are a few common challenges I see that keep popping-up in the Atlanta Java SaaS startup world. I’ll try my best to summarize what I think is the best-of-breed combination of SaaS startup technologies and hopefully arrive at a decent stack. These are common threads I keep seeing, and also tools that I used one of my many SaaS jobs or while I worked at JBoss.

read more »

I’d hate to call this an experiment as it deals with living creatures, many of them plucked from the ocean and thrown in to my new saltwater tank in the hopes they survive, but it is my first attempt at keeping a saltwater tank – so in essence – an experiment.

So far the inhabitants of this tiny ecosystem are:

1. Two Clownfish. (Laurel and Hardy)
2. One Peppermint Shrimp. (Pepe)
3. One Haitian Anemone, also known as the Pink-Tip Condy.
4. 5 Hermit Crabs.
5. 2 Turbo Snails.
6. Thousands of little copepods floating around being eaten by Laurel and Hardy.
7. An Asterina starfish (that I did not buy) missing 4 of his arms.
8. A bristleworm (that I did not buy). This thing is hideous and I wish I could kill it, but it does serve a purpose scavenging.

The Good News

• Almost two months in to it, no one has died, and new life/creatures are springing to life from the rock and sand bed.
• Kids love it – although the initial awe has worn off, they’re always finding something new moving around.
• Hermit crabs and Turbo snails mow through the initial algae spike like a fat guy at an all you can eat buffet.
• Maintenance is no more time-consuming than my freshwater tanks.
• Luckily I had a lot of the initial equipment, keeping costs low.
• Live Sand was given to me by Sal at AllReef in Acworth, GA, which I would think he pulled from his tanks. This surely helped speed up the initial cycling process.

The Bad News (Or what I’ve learned so far)

• “Nothing good happens in nature quickly. Only bad thing happen quickly in nature.” After you set everything up, you have to sit around for at least a month until the Live Rock “cures” and your ammonia dissipates and your nitrites lower and your nitrates rise. There’s an elaborate chemical cycle that has to complete before you should add even a snail. It’s too much to go in to in one blog post – books have been written over this subject.
• Lighting, arguably the most important piece of equipment for coral growth, is extremely expensive. A 24″ T5 High-Output fixture (with blue LEDs for night-time awesomeness) will run you about $90. Metal Hallide, the gold standard for coral growth, will run you your left arm and part of your left leg.
• Something I wish I would’ve known: Apparently the smaller the tank in this hobby, the greater the chances of it completely failing (ie, everything dying). Chemical changes in a large tank simply dillute faster over a larger area.
• To successfully keep a reef tank that is in the 100+ gallon range, you’re talking thousands of dollars on initial setup, and on top of that try and figure out how you’re going to handle water changes or topping off water, because you can only use water that is distilled/reverse-osmosis. My 5-gallon buckets won’t cut it.

I was recently looking for a Cache Control Filter for a Tomcat instance in an effort to speed-up one of my sites serving content. Remembering that I once lifted sample code written by Scott Stark at JBoss. Looking through my libs, I did find his original code with some adjustments I had made for including Expires tag for cross-browser compatibility in caching content for a certain amount of time. I figured I would post a copy, for any of you wanting to add header variables to Tomcat content output.

Note: I normally create two copies of this filter. One for text content like HTML, CSS, and JS files, and another for image content like GIF, JPG, PNG. The reason is because you may want images to be cached for a longer period of time in the user’s browser, as they don’t change as often as markup and code, and you don’t want to hack at your filter code to determine what type of content it is you’re serving. My $.02.

UPDATE: It has come to my attention that Tomcat 7 bundles a cache filter you can find here. More importantly, there is a standalone cache-filter on google code that is actively maintained, here.

read more »

Over the past 10 years I’ve spent working with multi-tenant software architecture, I’ve seen a lot of clever ideas and collosal mistakes made in designing systems for fault-tolerance and scalability. I decided it was time to note some of the best/worst practices here regarding the deployment of SaaS multi-tenant MySQL database designs, as food-for-thought to others designing a system from the ground-up.

Inevitably one of the initial crossroads a design team will reach is whether to design a system using one global schema or many schema. To put it simply, you are deciding on whether you want every customer’s data in on MySQL schema or you will have one-schema-per-customer within a MySQL instance. This is not a trivial design question, as getting this “right” in the beginning will cost you less headaches in the end. I’ll assume you aren’t designing in a vaccuum, and have a good idea of what the business side of your company would like to achieve with the software you’re deploying, ie. customer numbers, size of user-base, transactions per month, etc…

To make this simple, I’ve broken down some of the pros and cons of designing a multi-tenant database architecture with MySQL using one schema…

read more »

Those that know me personally, know my love for fish. It is likely amplified by the fact that I was born and raised in Miami, yet now find myself land-locked in Atlanta with the nearest ocean a 5-hour drive away. I have several fresh-water aquariums (Asian 30g planted tank pictured here) in-house, but this late-summer I decided to take my love for fish outdoors, by building a pond (in a rather muddy and useless part of my backyard).

The goal was to build a pond on a strict budget (I’m cheap!), thus requiring me to build most of it by hand, using the least amount of store-bought parts as possible. So I started digging…

… and I didn’t stop until the hole was 3ft at its deepest point, with several shelves sculpted along the edges at varying depths. Georgia clay is not Miami sand. Digging this 1500gallon hole took me 2 full weekends or 4 days of non-stop pick and shovel work, but once it was complete, it was time to place the under-liner. I found some Uhaul padding cloths that would work nicely, from the garage…

I didn’t skimp on the liner itself and bought 45epdm liner at the Atlanta Water Garden store ($270!) along with a 1800g/hr submersible pump ($150). I looked at the liners available at Lowes/HDepot, and didn’t want the low quality/price, because one hole in a liner would be a mess to fix. Once the liner was in place, the real work/engineering started on the filter. I wasn’t going to spend a minimum of $300 an a filter and another $300 on a skimmer, so I went the homemade bio-filter route. Using the Skippy site and this crazy british guy’s site as a guide, I built my own filter using a plain-old rubbermaid tank, pvc tubing, pvc valves, lava rock, and scotch-brite-like scrubbie pads (Dollar-store – $10). The lava-rock and scrubbies are used as a “house” for all that beneficial bacteria that will live there and eat-up all the junk that algae likes to feed on.

I also added several plants, like Anachris, Water Lettuce, two-leaf and four-leaf water clover. The plants will also compete with the algae and provide oxygen to the fish, so they’re not gasping for air. Within a few months, I should have crystal-clear water. Total cost for the filter was $50. Plants ran about $20.

Once the pond was filled, I bought some cheap-o fish… 20 minnows and 20 goldfish – total cost $7. Most people use these as feeders, and I wasn’t about to spend $200 on a koi!

I’ve done some work on the tubing since I filled the pond, painting all visible parts black with spray paint and burying most of the water-routing. I also got a great deal on stone-work for the edging from Lowes.

Although the original design called for the shape to be of a kidney-bean, somehow when I started digging it turned in to an oval. Luckily the shape works well with the location of the waterfall and pump, helping circulate water around the planted plants, so there aren’t any dead spots for algae to take hold and junk to collect.

Things left to do…

• Landscaping the area around the pond and create a viewing/sitting area.
• Purchase a palette of slate for the waterfall. It is currently using rocks I found during the Big Dig.

As expected, there was an algae bloom within a week of the pond being filled. It’s starting to subside now as the bacteria and plants take hold. I’ll post pictures next month and hopefully, I’m not staring at pea-soup.

The only design flaw thus far, I didn’t account for, was water overflow. It’s been raining a lot here lately, forcing me to open the drainage valve under the filter manually to drain the pond. If the water breaches the walls (think New Orleans levies), things can get ugly really fast, so building some automated overflow in the system will likely be a winter task.