Adam Howitt's Blog

May 27
2005

SQL Pivot by Date

Sometimes in CF I'm faced with the challenge of constructing something resembling a schedule of events over a range of arbitrary dates.  The goal is really to display all slots whether available or not.  One way is to query the database for all matching events during the date and have the code to loop over each slot and if there is something to fill it, put it in there, otherwise do something different like make a link available.

An alternative approach is to have the SQL return a row for each date and each resource you are considering, regardless of availability.  

This is a little more challenging than you would think since you have no master date table to LEFT JOIN on.  Here is a timesaving snippet to help.  It's a UDF for SQL Server 2000 which, when called with a start date and a number of days to return will give you the master date table.  Since it is a TABLE FUNCTION, any stored procedures calling it will not be forced to recompile (and slow down your app) because it avoids the creation of temporary tables.

SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS OFF
GO

CREATE FUNCTION dbo.fn_getDays (
/*
WRITTEN BY Adam Howitt http://www.adamhowitt.com
FUNCTION getDays
WRITTEN FOR SQL Server 2000
DATE 05.27.2005
Usage:
select dt
from fn_getDays('05/27/2005',5)
*/
@st_dt datetime,
@num_days int )
RETURNS @dateTable TABLE ( dt datetime not null)
AS
BEGIN
    WHILE (@num_days > 0)
    BEGIN
        SET @st_dt = dateadd(day,1,@st_dt)
        INSERT INTO @dateTable (dt)
        VALUES (@st_dt)
        SET @num_days = @num_days - 1
    END
    RETURN
END

GO
SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS ON
GO
Execute this in a database where you wish to use this functionality and then you can get results for your schedule as follows:
select resource_name,dt,event_name
from fn_getDays('05/27/2005',5) d
CROSS JOIN tb_resources r
LEFT JOIN tb_events e
ON d.dt = e.event_dt
You simply loop over the resultset in CF and if the event_name is empty display a link to book the resource.

CSS and Italics / Emphasis tags

I just spent an hour looking thru stylesheets and HTML switching between IE6 and Firefox looking for the cause of my layout woes.  The two browsers were rendering the same piece of content very differently.  A div was wrapping below the sub-navigation in IE6 but displaying fine in Firefox.

Pure coincidence had one of my coworkers walk past my cube to the window by my desk while I was in the middle of my search and he saw it immediately.  On a previous project they had tracked down a formatting problem.  Essentially, the presence of some HTML where text surrounded by an emphasis tag wraps onto a new line, IE increases the width of the containing box by 1 or 2 pixels but firefox *correctly* lets the errant text lean outside of it's container.

The fix for me was to reduce the width of the parent div containing the errant text by 2 pixels to allow it some breathing room in IE.  I scanned all the remaining layouts and it all looks fine.  Crazy stuff though. 

May 24
2005

Music Relay

Brian has passed me the baton, so here we go with my leg of the relay:

Total volume of music on your computer:
12.78 GB. 3336 songs.

The last album you purchased was:
"O" - Damien Rice

Song playing right now:

So Easy - Royksopp 

Five songs you’ve been listening to a lot recently, from several genres:

P.I.M.P. - Fifty Cent

I've had a few parties lately and I'm not sure how this crept into my party shuffle so often on iTunes but hey, it's up there.  It also used to be the ringtone for my fiancee on my cellphone since she was the one who introduced me to 50 cent originally.

Green Eyes - Coldplay

Despite my inability to get presale tickets, I still love listening to them.  I saw them at the Reading Leeds festival the year before I left for America and then at a small venue called the Metro with a couple of hundred people in Chicago.

Come - The Kleptones

Not commercially available this mashup team really rocks.  The first album I heard was "A night at the hip hopera" and subsequently "From Detroit to J.A.".  Brilliant.

Everybody's Changing - Keane

Just saw these guys for the second time in 3 months here in Chicago.  The Riviera venue was better musically but the Aragon ballroom was crazy because it looks like a castle!  The performance was not as good though and a little flat despite the presence of the cameras recording the event for the DVD.

Old Love - Eric Clapton

I've been learning a ton of old songs in guitar class and this is my favorite so far.  The chords are a little more novel than the usual tripe and the sound is really funky.

Five people to whom you’re passing the baton:

Bugger. Five bloggers who may take the baton from me who haven't already done this:

Cameron Childress, because he knows the way to San Jose. (well, San Diego but I don't know any songs about SD). 

Dean Saxe, because he came to visit Chicago recently. 

Sean Corfield, for his sagely words of advice in response to my request for reading material for my new position as Senior Architect. 

Robert Occhialini, for hosting Melissa and I this weekend as we float in to Atlanta to celebrate Mark's wedding.

Spike, for pushing CFEclipse onto my desktop and increasing my productivity. 

Bugger again.  Just saw that Sean and Spike already did this. Guess I'll leave it at three. 

Chicago ColdFusion Birthday Party

I just read Damon Cooper's post about ColdFusion turning 10 years old and hopped over to the Macromedia event details.  It seems they have a San Fran and Boston event planned so far.  Would anyone be interested in a Chicago event to cover the midwest programmers? 

I am volunteering my services to arrange the whole thing and have asked Ben Forta if he would be able to attend since he lives in this neck of the woods, although I'm sure they already have him committed to one of the other venues.  It would be on July 13th and timed to coincide with the East and West coast parties.

If you could give a tentative RSVP in the comments it will help me pursuade Macromedia that we need a midwest event.

May 23
2005

Another Google Theory Bites the Dust

Another client was experiencing similar symptoms to Pittman Guitar Repair: it seems like some part of the page is causing the Google spider to stop reading the page or adding links into the index when it hits the first page.

This gave me the ability to look at what is present on my own blog and compare the HTML to that of the two sites not being spidered.  One thing I noticed was that the DOCTYPE was common to the two sites ignored by Google while mine was missing a DOCTYPE altogether and hence defaulting to HTML 4.01 transitional.

The Theory

The XHTML doctype was causing Google to choke if the document was not valid XHTML.  For reference, this is the DOCTYPE in question.

 

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

 

Method 

I carefully crafted a six model experiment and linked to it to send in the spiders.  The first 3 models were designed to contain valid XHTML while the last 3 were missing some closing tags and added an unknown tag called badtag which didn't close correctly. 

Within each group of three models, I had one setup with no DOCTYPE at all, a second with the HTML transitional 4.01 DOCTYPE and the last with the XHTML DOCTYPE above.

Each model pointed to it's own results page with the same characteristics as the model (valid/invalid and the same DOCTYPE).

Goals 

After the models had been added to the index, if the theory held true, the results pages for the malformed XHTML pages should have been the only ones not in the index.

Results

The root model pages all appeared in Google within a day of the example (due to a sneaky link in my footer) and the destination pages linked from each model appeared shortly after.

Put simply, I was wrong!  It seems the DOCTYPE has absolutely no bearing on Google being able to spider the site if the document does not conform. 

Conclusion

I still have no idea why Pittman Guitar Repair isn't getting spidered beyond the first page but I do know what isn't causing a problem:

  1. Long URLs -  Knowmysize.com has no issues getting indexed daily and uses an even longer link strategy.
  2. UUIDs in the URL - Reference point 1
  3. CFM pages instead of HTML pages - Reference point 1
  4. Links embedded in an unordered list HTML element - this was the point of my previous experiment on unordered lists and Google.
  5. Document doesn't parse based on the DTD/DOCTYPE selected - the point of this post.
  6. Fusebox 3 - This is covered in the comments of this earlier post.
  7. Inbound links - Both these sites have at least 2 inbound links from similar sites.
  8. Transparent PNG hack - Pittman Guitars used to use the special hack for displaying 24bit transparent PNG images so I took it out but to no avail.
If anyone has any other ideas, I'm all ears but in the meantime, I've taken out the DOCTYPE until the Google Cache date for the home page indicates they have visited since the change since there may have been a flaw in the experiment in that the models didn't use the same page HTML.  I'll update the post if anything changes.

May 18
2005

Organic Vegetables Strike Back

In an outstanding piece of timing and marketing Free Range Media has produced Grocery Store Wars for the Organic Trade Association.  It is a Flash Movie based on Star Wars but with the names changed to protect the innocent.  Starring Cuke Skywalker, Obe Wan Cannoli and many more I won't spoil for you.  Missing characters I would like to see were "Boba Feta Cheese" and Jabba the Pizza Hut.

Can't wait to see Episode III this weekend...

Most Amusing PayPal Scam Email

I received yet another PayPal scam email this morning but when I read closer I started to think this might have something to do with Dan Brown's Angels and Demons.  The clue is in the misspelled company name "Congratulations! Papal".  I have changed the URLs in case somebody is tempted to login.  So the darwin award for Spammers and Scammers goes to gauroipstepdm@263.net.cn:

PayPal


Dear valued PayPal® Member

It has come to our attention that your

Congratulations!Papal®

account information needs to be updated as part of our continuing commitment to protect your account and to reduce the instance of fraud on our website. If you could please take 5-10 minutes out of your online experience and update your personal records you will not run into any future problems with the online service.

You must

click the link below and enter your password on the following page

to Update your email address.

http://83.246.118.73/paypal.com/..index.htm


However, failure to update your records will result in account suspension. Please update your records on or before:

  • Buy from an online auction
  • Pay on a merchant

    website

  • Send money to anyone with an email address




You can also confirm your However, failure to update your records will result in account suspension. Please update your records on or before: email address by logging into your PayPal account at http://83.246.118.73/paypal.com/../index.htm. Click on the "Confirm email" link in the Activate Account box and then enter this confirmation number: 1328-8599-4830-9811-3174

Thank you for using PayPal!

The PayPal Team



PayPal Email ID PP468

Protect Your Account

Info

Make sure you never provide your password to

fraudulent websites.

To safely and securely access the PayPal

website or your account, open a new web browser (e.g. Internet Explorer or Netscape) and type in the PayPal URL (https://www.paypal.com/us/) to be sure you are on the real PayPal site.

PayPal will never ask you to enter your password in an email.

For more information on protecting yourself from fraud, please review our Security Tips at https://www.paypal.com/us/securitytips

May 13
2005

Glazer to take Manchester United

As Malcolm Glazer, owner of the Tampa Bay Buccaneers American Football team, closes in on full control of Manchester United I have a dream about his intentions.  I have been a Manchester United fan since my father took me to my first game in 1979 and they were the number one reason I played football as a child and still play each weekend to this day.

My dream is that Malcolm Glazer can take Football (soccer) to the next level in the US through his marketing experience.  The action plan for me would be as follows:

  1. Wrangle the rights to the English Premier League from the grasp of Setanta. They have done nothing to promote the sport in the US.  Charging extorionate fees to bars to show the game discourages anyone who can't see the business advantage from showing the games.
  2. Subsidize or work with the major US TV networks like FOX, NBC, ABC or ESPN to win the contract. Show the games each week for free across America and take advantage of the obvious ratings soccer would pull in.  Saturday morning is when millions of American children get up and sit in front of the TV to watch cartoons.  That is when the games are shown (9am CST).  The most compelling reason this makes sense is that Soccer is the number one sport played in America by children.
Why should this be a dream for Malcolm Glazer to execute?  Marketing dollars.  To the lucky network that wins the contract there would be a vast loyal following which means more marketing dollars for them too.  So come on Setanta.  Give up your hold on the English Premier League and let a network who can advance the promotion of the sport in the US take control.

Database Analysis

I was migrating a database between dev and stage servers today and had to clean out the test data we were using.  Although we had a backup, I didn't want to blow away any of the other team members data which should be persisted.  Typically on apps I have worked on from scratch I'd insist that any data which needs to exist in the database should be included in a build script as inserts but this app has been in development for 3 developers so it is beyond the point of recognition.

I wrote a simple cursor using SQL Query Analyzer's READONLY Cursor template to loop over the schema for the database and return the number of rows in each table.

SET NOCOUNT ON
DECLARE find_rows CURSOR
READ_ONLY
FOR SELECT table_name
FROM INFORMATION_SCHEMA.TABLES

DECLARE @name varchar(1000),
    @stmt nvarchar(1000)
OPEN find_rows

FETCH NEXT FROM find_rows INTO @name
WHILE (@@fetch_status -1)
BEGIN
    IF (@@fetch_status -2)
    BEGIN
        SET @stmt = N'select '''+@name+''' as tbname, count(*) FROM '+ @name
        exec sp_executesql @stmt
    END
    FETCH NEXT FROM find_rows INTO @name
END

CLOSE find_rows
DEALLOCATE find_rows
GO
So the obvious next step is how to delete that data from a large database?  This particular app has close to 100 tables.  The modification to the cursor is easy.  Replace the SET @stmt and exec sp_executesql lines with the following snip to get a generated delete script:
PRINT 'DELETE FROM ' + @name
PRINT 'GO'
Copy the generated output to the clipboard and you can paste it in a new query window.  Throw in a BEGIN TRAN before the whole script and run it.  The key here is to look for foreign key constraints which have prevented deletions from taking place.  If you see any, in the generated output, find out which table it relies on and reorder the delete statements to resolve the constraint conflict.  IMPORTANT: Execute the ROLLBACK TRAN statement to put all your data back in place that you just deleted!  You can then execute the delete script again and until you have all your dependencies ironed out.  Now it is time to test your app to make sure the absence of data doesn't break the layout or god forbid, break the code but that's a different post!

May 06
2005

Motorola / Comcast DVR Mute Problem

I have a DVR from Comcast and it has a software bug (they all do apparently) where the remote isn't pre-programmed to handle the muting of the cable box, just the tv itself. 

This becomes a problem when you leave your DVR powered off when it needs to record something. It will happily power on and correctly records the show but when it turns itself on, the volume is muted on the cable box.  I can see the engineers thinking - we don't want to wake people up in the middle of the night when it starts recording so we'll start it in mute. 

Jeff was able to find a tip which I'll share below for resolving the issue:

  1. Press the "Cable" button at the top of the remote to put it into Cable Box control mode.
  2. Press and hold the "Setup" button until the "Cable" button blinks twice.
  3. Enter code 994. The "Cable" button will blink twice.
  4. Press (do not hold) the "Setup" button.
  5. Enter code 00141.
  6. Press whatever button you want to map the mute function to.

Thanks Jeff! 

More Entries