Adam Howitt's Blog

Sep 04
2009

Ugly MySQL tools for Mac AKA I hate Navicat

Seriously. Why should I be handicapped for trying to write SQL on a mac client? I've tried Navicat and hate it with a passion. For an OS that prides itself on pretty interfaces the tools for working with databases I've seen are all really ugly. MySQLYog for PC ain't no oil painting either but it embodies the important pieces of a SQL tool. Here's what I want - if anyone has any tips on tools, paid or free - I don't care:

1. Connection and object browser on the left. Drill down into tables to see column names, indexes, foreign keys, triggers. Navicat inconveniently makes you right click on the table name to view any of these details. When I'm writing a query I don't want to have to open a new tab to show an ugly list of table columns I can't refer to while I'm writing code without switching tabs.

2. SQL Editor windows with a split results pane that doesn't vanish if I edit one piece of a query. Navicat hides the SQL results if you change a query. Seeing sample data as I work on a query REALLY helps, especially since Navicat won't simultaneously show column specs (ref earlier point).

3. Edit stored procedure or views like I just wrote them. In every other SQL client I've used I right click a view or stored procedure to see the SQL stored in the database, preferably formatted. Navicat presents views as one long string of unformatted text and for stored procedures it offers an edit window for the inner section of the procedure but hides the implicit create statement and/or drop statement.

4. Right click ANY object to see a SQL script representing the object.

Is this too much to ask? I've tried running SQLyog in a VM but hate hogging so much memory to run a SQL client, not to mention the fact I routinely have issues authenticating my VM and host machine simultaneously on Starbucks' wi-fi.

Nov 29
2007

A New Twist for Hierarchical Data

If you've ever had to work with a database model where rows of data represent hierarchical relationships then SQL Server 2005 has a hidden gem you may be interested.  Here are some domain examples to help you spot the type of problem this can solve:

  1. Tasks: where a task may be part of a larger task or can be made up of subtasks
  2. Bill of Materials: in a manufacturing environment where an item is made up of many other component items
  3. Family trees: Father has a Grandfather and Children but all are people
  4. Company Org Chart: the company org chart where employee Y is managed by employee X and manages employees z1,z2 and z3
Many ways to skin a cat
Before I introduce the old approach here are a few ways you might have handled it.  
  1. Joe Celko's nested set model: don't be surprised if you haven't heard of this approach - it's generally acknowledged as the fastest way to access hierarchical datasets but the complexity of implementing it scares many programmers away.  Using this approach requires you to re-architect the way the data is stored, accessed, inserted and updated so if your database already exists this would be a big undertaking.  If your data is both wide and deep, and you routinely access the complete dataset then read one of Joe's many articles on nested sets or buy his book "SQL for Smarties".
  2. Recursive stored procedures or functions: write a stored procedure or function to cursor over the children of the node you are looking at and then recursively call itself to go deeper in the recordset to build a table (need to write a function) or process each row (need to use a stored proc).  
The new rock and roll: recursive queries
The new feature of SQL Server 2005 is that it supports recursive queries.  It replaces the need to write a recursive table valued user defined function with a construct to allow you to quickly access the recordset you need to work with.  This recordset could then be returned by the procedure or used to drive a cursor to iterate over each node in the sub-tree in a particular order.

Enough talk - here is a concrete example based on the family tree notion:
WITH descendants(personid, parent_personid, generation) AS (
    /* Get the base case */
    SELECT personid, parent_personid, 0 as generation
    FROM family_members
    WHERE parent_personid IS NULL
    /* Union the recursive case */
    UNION ALL
    /* Find all family members (fm) who are descendants (d) of someone already in the descendants temp table */
    SELECT fm.personid, fm.parent_personid, generation+1
    FROM family_members fm
    INNER JOIN descendants d
    ON fm.parent_personid = d.personid
)
SELECT personid, parent_personid, generation
FROM descendants
/* Order the rows by generation ascending so that grandfather comes before father */
ORDER BY generation ASC


What is going on here?
The WITH table_name(column_list) AS (QUERY) construct allows you to define an inline view and can be used for non-recursive purposes too but in this case it defines a base case and then joins against itself using temporary tables until there are no more joins to be made or a criteria is met (we could limit it to WHERE dateDeceased IS NOT NULL to find only deceased generations).  The first generation is the base case and represents our starting points in the tree. Note that this example will grab everyone who has no parent but we could just as easily have asked for a particular ancestor and found descendants of that person.

Still no clearer?
If our table family members contained 3 rows:

Person  Parent_Personid   Name
1       NULL              Grandfather
2       1                 Father
3       2                 Son

We could represent this with SQL as shown below but this only works for the top 3 levels.  Clearly this would be unmanageable for more layers.  This is very similar to the way the recursive query works - it builds each recordset working it's way down adding rows to a temp table and then using that temp table to get the next set of rows to add.
SELECT personid, parent_personid, 0 as generation
FROM family_members
WHERE personid = 1
UNION
SELECT personid, parent_personid, 0 as generation
FROM family_members
WHERE parent_personid = 1
UNION
SELECT personid, parent_personid, 0 as generation
FROM family_members
WHERE parent_personid IN (
    SELECT personid, parent_personid, 0 as generation
    FROM family_members
    WHERE parent_personid = 1
)


Performance Counts
This approach will work as fast, if not slightly faster than any other recursive approach.  As Steve Nelson told me yesterday - there is no magic going on here.  What it does is give you a more convenient way to write and think about recursive queries.  If you are looking for < 10ms calculation times all the time at any cost then the Celko way is the way to go.  If you write some use cases for your application and test this approach it may be that you are filtering the dataset enough to get great execution times anyway.  For example, in the family example we are interested in just one ancestor, the query times shouldn't be much slower than the Celko approach but if you are routinely looking for all people with no parent_personid for vast datasets then you will see big performance problems. As always, test each approach yourself and find out what works for you.

Sep 10
2007

Profitable Online Marketing with Google Analytics

After 9 years of working for other people I've decided to go solo and start my own consulting business.  The site is rudimentary right now as I only handed in my notice today.  As regular readers may be aware, I've evolved from an entry level programmer to a Senior Application Architect in my current role since getting my Masters in Software Engineering and developed an interest in implementing Google technologies.  In my spare time I've been fostering a worldwide smash-hit, WalkJogRun.net, with over 4,000 visitors per day and 100,000 routes worldwide.  I'm addicted to Google Analytics and have a white paper due out under my current employers, Duo Consulting, explaining how to monitor the effectiveness of your marketing campaigns and tune the content on your website.

The last three years at Duo has been a fantastic experience for me giving me the freedom to explore exciting new technologies and all under billable hours because our clients are willing to try new things.  I've load tested and refined the Chicago Park District programs website to support as many as 3,000 transactions in the first 3 minutes of registration every quarter, debugged and load tested third-party ColdFusion CMS tools and built a Cicada Emergence Google Mashup in the last year alone.  I've implemented Google Analytics for many of our clients resulting in important business changing results.  Clients have paused underperforming ad campaigns, monitored the performance of redesign work, reallocated advertising dollars and even redesigned their shopping cart checkout process on the strength of tell-tale numbers through my analysis.

The primary focus of my new business will be helping clients to install Google Analytics, establish their goals for the websites and then monitor the effectiveness of their sites and the advertising campaigns they create.  I'll help them identify areas of their sites underperforming and evaluate their ad campaigns to get the most out of every dollar.  I have colleagues who can help with content development, Duo will be my first referral as far as deeper restructuring work and I can do development work too.  

The secondary focus for my business will be to load test web applications and either help developers benchmark the sites or discover the bottlenecks and resolve them.  I see this as a huge opportunity as the web continues to grow and web applications surpass the lowly traffic expectations they were initially conceived to handle.  In some cases the trouble will be at the database level, sometimes it is the JVM, sometimes it's an innocuous ColdFusion funciton like createUUID that hit the OS behind the scenes and crash JRun every 10 concurrent visitors.  
If you have read any of my posts in the past and value the expertise you have found, or if you have read this description and think you or someone you know would value from this kind of service, please pass on the link or get in touch.  I can be reached by email at adamhowitt@gmail.com or by phone at +1 (312) 714 9229.  Thanks for reading and stay tuned for more of the same tips, tricks and commentary.

Apr 30
2007

See Howitt Run

I have 2 weeks left to raise the remainder of my $3,900 goal for Leukemia and Lymphoma and I need your help. I'm in Week 15 of the Team in Training plan to run the San Diego marathon on June 4th.  Saturday was an 18 mile run out in Darien, IL and my legs are still tight from the challenge.  It's a distinct change from the flat course along Chicago's lakefront.  We've got a 10 mile run this weekend before the next jump to 20 miles the following week.

I was initially running in memory of my Aunty Moreen who passed away in 2004 after a 6 year struggle with bowel cancer.  The weekend after I started my training we discovered that my wife's 3 year old niece Haidyn had been diagnosed with ALL (Acute Lymphoblastic Leukemia).  Over the past 12 weeks she has been a tough little girl fighting infections caused by the drugs reducing her immune response and dealing with the loss of hair caused by the chemo.  At mile 16 on Saturday as I felt like my legs couldn't keep going I remembered the pain her and her family have experienced and it pushed me to the end.

If you have ever read one of my posts on SQL, AJAX, ColdFusion or even my most popular post on how to fix a bug on your Motorola DVR, please take time to donate.  I appreciate anything you can spare.

Mar 22
2007

When a NULL is not a NULL

I've just come across a little logic flaw in a SQL construct I have relied on for some time.  Needless to say I have some work ahead of me to find places I have used it and fixed it. 

To keep my stored procedures tidy but still allow optional parameters (like CF can deliver by using CFIF around the extra WHERE clause), I have been using this type of thing:

    SELECT *
    FROM any_table
    WHERE field_id = ISNULL(@field_id,field_id)

This code is supposed to work as follows - give me all rows where field_id in the row matches the passed in parameter @field_id but if the @field_id wasn't passed in was passed in as a NULL show all rows. It shows all rows because in all cases the field_id in the row should match itself, correct?
WRONG!  If field_id is actually a NULL value then the row becomes WHERE NULL = NULL.  I just ran some tests and was baffled but then remembered the rules of three valued LOGIC (where there are three possible values of true, false or NULL) - NULL compared to anything is always an unknown value.  This article gives it a good wrap:

After realizing my mistake the code now looks like this to achieve the same effect but correctly:
    SELECT *
    FROM any_table
    WHERE field_id = @field_id OR @field_id IS NULL


Hope that helps someone.

Feb 09
2007

ColdFusion Development

Despite the bottomless pit of information available in Google, I find myself falling back on a core set of sites for the answers to my questions.  Through the magic of Google Co-op I have created a ColdFusion Development Search Engine.  It is a hand made list of sites I trust for answers to my ColdFusion questions.  I've also included and tagged JavaScript, SQL, Fusebox and AJAX resources with the idea that anything related to CF that I use on a daily basis to do my job is fair game.

If you can't find what you are looking for, let me know which site provided your answer and I can add it to the list after reviewing it.   

UPDATE: The URL for the search is http://www.webdevref.com 

Jan 09
2007

Help with OO Problem

I've studied OO at University in the shape of Eiffel, I've programmed OO somewhat lightly in Java and developed OO in CFCs but there are some classes of problem I just don't get.  If anyone reading this could offer insight from their own experience that would be great!  I'll offer problem 1 for now and dependent on the response I might hit you up for problem 2.

"is related to"

I've called this "is related to" because I'm talking about the modelling of relationships between classes.  This comes in the context of how to use objects in a Content Management System.

For example, lets say I have a website with an about us section and in that section, an programmer list with a list of the key skills of each programmer.  There is also a "key skills list" which has a list of programmers who are in posession of the skill.  Programmers own skills but they aren't necessarily composed of those skills e.g. "has a" and this isn't inheritance since programmers aren't a type of ColdFusion.

So, to make it real, under the list of programmers I have Ben Forta, DHH, Adam Howitt, Sean Corfield and Cameron Childress.  If you view the detail for each programmer you will see that all but DHH have ColdFusion experience.  All have MySQL experience and lastly Adam and DHH have dabbled in Rails.  If I navigate to the ColdFusion page I see a list: Adam, Ben, Cameron, Sean with a brief snippet of the full bio under each.

On the ColdFusion page I would fetch the ColdFusion object and dutifully write out each attribute before getting the related programmers:

<h1>#skillObj.getTitle()#</h1>
<p>#skillObj.getBody()#</p>
<h2>Related Programmers</h2>
<ul>
<cfset stProgrammers=skillObj.getProgrammers()>
<cfloop collection="#stProgrammers#" item="programmer">
<cfset currentProgrammer=stProgrammers[programmer]>
<li>#currentProgrammer.getFirstName# #currentProgrammer.getLastName#<br/>
<p>#currentProgrammer.getShortBio()#</p></li>
</cfloop>
</ul>


My question is how do you implement getProgrammers and what do the classes look like?  My initial thought is that there is a Programmer class and a Skill class.  Now for the chicken and the egg: does a programmer has a struct of skills (order isn't important) or does the skill have a struct of programmers (again, order not important).  Or both?  What do I store? The struct of class instances (programmer objects) or the struct of ids?  If I use a struct of objects, the call getProgrammers has all the information it needs to print this page.

If I store the ids of the objects then I increase cohesiveness by requiring the method to call a function to instantiate the skill class for each id. Storing the ids creates a new issue - I have to load each skill one by one, making multiple database calls for each page request instead of a single database call to load all related skills.  In the current system, I load all skilll attributes with one query, all related programmers with another then display them to the screen.  

The biggest issue I am having trouble understanding is what gets stored in memory.  We don't really have a Programmer or a Skills page, but model the employee structure and skills profiles of clients in the service industry.  Some clients have over a thousand employees with experience or skills selected from a list of over a hundred.  Adding up the potential number of objects, some clients may have over 5,000 content objects which may be called by any number of pages at a given time.  The current procedural approach works but we're trying to see if there is a better OO design to fit the purpose.

Nov 08
2006

JDBC Drivers 3.4 Bug for ColdFusion

Not only did Adobe technote http://www.adobe.com/go/1a3c2ad0 not fix the issue I was having with quotes doubling, but it introduced a bug into all of our applications built from our CMS codebase for SQL Server.  I'll pre-empt some comments by saying that I know the following approach is non-standard and there is a workaround but the fact is that it was done and we are looking to fix it.  When we insert rows into tables with uniqueidentifier types we use triggers to return the generated UUID back to ColdFusion in some cases.  For example:

INSERT INTO tb_content_history(history_id,etc..)
(SELECT NEWID(), etc.)

For the curious, the alternative would be to use the SQL variable @@identity which only works on columns with a single identity column (not combined keys):

INSERT INTO tb_content_history(history_id,etc..)
(SELECT NEWID(), etc.)
SELECT @@identity as history_id
 

The bug itself is that the first code sample will fail to deliver a recordset so if your cfquery name was qAddHistory you get an error message if you try to dump out qAddHistory to the screen since the recordset is empty.  I'm not sure if this change in behaviour was deliberate or incidental but either way, my options are to find and fix every instance of the old style code to the @@identity approach or rollback the driver.

For anyone wishing to try the trigger approach here is some example code

CREATE TRIGGER [dbo].[tr_history_id] ON [dbo].[tb_content_history]
FOR INSERT 
AS
SELECT history_id FROM INSERTED

Oct 10
2006

Pagination made easier with SQL Server 2005

In the past recordset pagination with SQL Server was like standing on your head and trying to juggle.  I wrote a pagination stored procedure for SQL Server 2000 which takes the top x of the reversed recordset then the top y of that.  The trouble is that SQL Server 2000 doesn't allow the number of records to pass into the top command as a variable hence the silly EXEC for dynamic sql statements.

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

ALTER  PROCEDURE [dbo].[usp_cms_select_paging_amh]
/*********************************************************************************
--** Stored Procedure Name: usp_cms_select_paging_amh
--********************************************************************************
--**
--** Description:
--**    Alternative to standard CMS paging
--** Revision History:
--**    AMH 06.01.05
--**    Updated to account for functional sort criteria e.g. CAST(a.attribute_desc as varchar(2000) DESC
--**    Extended notes:
    I've fixed the paging query again.  The issue is caused when you pass the sort order argument a
    function as one of the fields to sort by, for example:
    CAST(a.attribute_desc AS varchar(8000))',
    Which was getting incorrectly interpreted due to the extra spaces (the mask it originally used
    was "alias.column_name sort_direction".  Now it correctly acknowledges the sort order in this case.  
--** Example:
DECLARE
    @fieldNames varchar(1000),
    @tableName varchar(1000),
    @orderby varchar(100),
    @page int,
    @numRows int,
    @orderbyField varchar(100)
SET    @page = 2
SET     @numRows = 2
SET    @fieldNames = '*'
SET    @tableName = '(select top 3 * FROM vw_attorney_biography) ab'
SET     @orderby = 'ab.name DESC'
SET    @orderbyField = 'name'
EXEC usp_cms_select_paging_amh @fieldNames, '', @tableName, @page, @numRows, 1, '', @orderby, ''
*/
(
    @strFields VARCHAR(4000) ,
    @strPK VARCHAR(100),
    @strTables VARCHAR(4000),
    @intPageNo INT,
    @intPageSize INT,
    @blnGetRecordCount BIT,
    @strFilter VARCHAR(8000) ,
    @strSort VARCHAR(8000),
    @strGroup VARCHAR(8000)
)
AS
BEGIN
    DECLARE
        @topPage int,
        @theSql varchar(2000),
        @theCount varchar(2000),
        @blnBringAllRecords BIT,
        @strPageNo VARCHAR(50),
        @strPageSize VARCHAR(50),
        @strTopPage VARCHAR(50),
        @strFilterCriteria VARCHAR(8000),
        @strSimpleFilter VARCHAR(8000),
        @strSortCriteria VARCHAR(8000),
        @strGroupCriteria VARCHAR(8000),
        @oppositeSort1 VARCHAR(8000),
        @oppositeSort2 VARCHAR(8000),
        @intRecordcount INT,
        @intPagecount INT,
        @errMessage varchar(3000)    
    --vars for sort reversing
    DECLARE @this varchar(255),
        @rest varchar(8000),
        @pos int,
        @this_alias varchar(100),
        @this_column varchar(100),
        @this_direction varchar(100),
        @num_rows int
    --vars for recordcount
    DECLARE @totalRecordCount int,
        @ParmDefinition NVARCHAR(500),
        @SQLString NVARCHAR(4000)
    --********* CALCULATE THE LAST ROW TO SHOW ***************
    SET    @topPage = @intPageNo * @intPageSize
    --******** NORMALIZE THE FILTER AND SORTING CRITERIA
    --if they are empty, we will avoid filtering and sorting, respectively, executing more efficient queries
    IF @strFilter IS NOT NULL AND @strFilter != ''
        BEGIN
            SET @strFilterCriteria = ' WHERE ' + @strFilter + ' '
            SET @strSimpleFilter = ' AND ' + @strFilter + ' '
        END
    ELSE
        BEGIN
            SET @strSimpleFilter = ''
            SET @strFilterCriteria = ''
        END
    IF @strSort IS NOT NULL AND @strSort != ''
    BEGIN
        SET @oppositeSort1 = ''
        SET @oppositeSort2 = ''
        --CREATE A STRING WHICH REVERSES THE SORT ORDER
        SET @this = ' '
        SET @rest = @strSort
        SET @pos = PATINDEX('%,%', @rest)
        WHILE (@pos > 0)
        BEGIN
            set @this=ltrim(rtrim(substring(@rest,1,@pos-1)))
            set @rest=ltrim(rtrim(substring(@rest,@pos+1,len(@rest)-@pos)))
            -- function(alias.column text)
            IF PATINDEX('%(%.%',@this) > 0
            BEGIN
                IF PATINDEX('% asc%',lower(@this)) > 0
                BEGIN
                    SET @this_direction = 'asc'
                    SET @this = ltrim(rtrim(substring(@this,1,PATINDEX('% asc%',@this)-1)))
                END
                ELSE IF PATINDEX('% desc%',lower(@this)) > 0
                BEGIN
                    SET @this_direction = 'desc'
                    SET @this = ltrim(rtrim(substring(@this,1,PATINDEX('% desc%',@this)-1)))
                    PRINT 'here3'
                END
                ELSE
                    SET @this_direction = 'asc'
                IF PATINDEX('%.%',@this) = 0
                BEGIN
                    SET @this_alias = ''
                    SET @this_column = ltrim(rtrim(substring(@this,PATINDEX('%.% %',@this)+1,PATINDEX('% %',@this)-PATINDEX('%.% %',@this))))
                END
                ELSE
                BEGIN
                    SET @this_alias = ''
                    SET @this_column = ltrim(rtrim(substring(@this,PATINDEX('%.% %',@this)+1,PATINDEX('% %',@this)-PATINDEX('%.% %',@this))))
                END
            END
            -- alias.column dir         e.g. SELECT PATINDEX('%.% %','ab.hat  this')
            ELSE  IF PATINDEX('%.% %',@this) > 0
            BEGIN
                SET @this_alias = ltrim(rtrim(substring(@this,1,PATINDEX('%.% %',@this)-1)))
                SET @this_column = ltrim(rtrim(substring(@this,PATINDEX('%.% %',@this)+1,PATINDEX('% %',@this)-PATINDEX('%.% %',@this))))
                SET @this_direction = ltrim(rtrim(right(@this,len(@this)-PATINDEX('% %',@this))))
            END    
            -- alias.column         e.g. SELECT PATINDEX('%.%','ab.hat  this')
            ELSE IF PATINDEX('%.%',@this) > 0         
            BEGIN
                SET @this_alias = ltrim(rtrim(substring(@this,1,PATINDEX('%.%',@this)-1)))
                SET @this_column = ltrim(rtrim(right(@this,len(@this)-PATINDEX('%.%',@this))))
                SET @this_direction = 'ASC'
                
            END    
            -- column direction         e.g. SELECT PATINDEX('% %','ab  this')
            ELSE IF PATINDEX('% %',@this) > 0
            BEGIN
                SET @this_alias = ''
                SET @this_column = ltrim(rtrim(substring(@this,1,PATINDEX('% %',@this)-1)))
                SET @this_direction = ltrim(rtrim(right(@this,len(@this)-PATINDEX('% %',@this))))
                
            END    
            -- column only         e.g. SELECT PATINDEX('%.% %','ab.hat')
            ELSE
            BEGIN
                SET @this_alias = ''
                SET @this_column = @this
                SET @this_direction = 'ASC'
                
            END    
            -- add this flipped column to the opposite sort list
            IF len(@oppositeSort1) != 0
            BEGIN
                SET @oppositeSort1 = @oppositeSort1 + ', '
                SET @oppositeSort2 = @oppositeSort2 + ', '
            END
            IF UPPER(@this_direction) = 'ASC'
            BEGIN
                SET @oppositeSort1 = @oppositeSort1 + @this_column + ' DESC'
                SET @oppositeSort2 = @oppositeSort2 + @this_column + ' ASC'
            END
            ELSE
            BEGIN
                SET @oppositeSort1 = @oppositeSort1 + @this_column + ' ASC'
                SET @oppositeSort2 = @oppositeSort2 + @this_column + ' DESC'
            END
            SET @pos= PATINDEX('%,%', @rest)
        END
        set @this=@rest
        -- NOTE: IDENTICAL CODE TO ABOVE
        -- function(alias.column text)
        IF PATINDEX('%(%',@this) > 0
        BEGIN
            IF PATINDEX('% asc%',lower(@this)) > 0
            BEGIN
                SET @this_direction = 'asc'
                SET @this = ltrim(rtrim(substring(@this,1,PATINDEX('% asc%',@this)-1)))
            END
            ELSE IF PATINDEX('% desc%',lower(@this)) > 0
            BEGIN
                SET @this_direction = 'desc'
                SET @this = ltrim(rtrim(substring(@this,1,PATINDEX('% desc%',@this)-1)))
                PRINT 'here3'
            END
            ELSE
                SET @this_direction = 'asc'
            IF PATINDEX('%.%',@this) = 0
            BEGIN
                SET @this_alias = ''
                SET @this_column = ltrim(rtrim(substring(@this,PATINDEX('%.% %',@this)+1,PATINDEX('% %',@this)-PATINDEX('%.% %',@this))))
            END
            ELSE
            BEGIN
                SET @this_alias = ''
                SET @this_column = ltrim(rtrim(substring(@this,PATINDEX('%.% %',@this)+1,PATINDEX('% %',@this)-PATINDEX('%.% %',@this))))
            END
        END
        -- alias.column dir         e.g. SELECT PATINDEX('%.% %','ab.hat  this')
        ELSE  IF PATINDEX('%.% %',@this) > 0
        BEGIN
            SET @this_alias = ltrim(rtrim(substring(@this,1,PATINDEX('%.% %',@this)-1)))
            SET @this_column = ltrim(rtrim(substring(@this,PATINDEX('%.% %',@this)+1,PATINDEX('% %',@this)-PATINDEX('%.% %',@this))))
            SET @this_direction = ltrim(rtrim(right(@this,len(@this)-PATINDEX('% %',@this))))
        END    
        -- alias.column         e.g. SELECT PATINDEX('%.%','ab.hat  this')
        ELSE IF PATINDEX('%.%',@this) > 0         
        BEGIN
            SET @this_alias = ltrim(rtrim(substring(@this,1,PATINDEX('%.%',@this)-1)))
            SET @this_column = ltrim(rtrim(right(@this,len(@this)-PATINDEX('%.%',@this))))
            SET @this_direction = 'ASC'
            
        END    
        -- column direction         e.g. SELECT PATINDEX('% %','ab  this')
        ELSE IF PATINDEX('% %',@this) > 0
        BEGIN
            SET @this_alias = ''
            SET @this_column = ltrim(rtrim(substring(@this,1,PATINDEX('% %',@this)-1)))
            SET @this_direction = ltrim(rtrim(right(@this,len(@this)-PATINDEX('% %',@this))))
            
        END    
        -- column only         e.g. SELECT PATINDEX('%.% %','ab.hat')
        ELSE
        BEGIN
            SET @this_alias = ''
            SET @this_column = @this
            SET @this_direction = 'ASC'
            
        END    
        -- add this flipped column to the opposite sort list
        IF len(@oppositeSort1) != 0
        BEGIN
            SET @oppositeSort1 = @oppositeSort1 + ', '
            SET @oppositeSort2 = @oppositeSort2 + ', '
        END
        IF UPPER(@this_direction) = 'ASC'
        BEGIN
            SET @oppositeSort1 = @oppositeSort1 + @this_column + ' DESC'
            SET @oppositeSort2 = @oppositeSort2 + @this_column + ' ASC'
        END
        ELSE
        BEGIN
            SET @oppositeSort1 = @oppositeSort1 + @this_column + ' ASC'
            SET @oppositeSort2 = @oppositeSort2 + @this_column + ' DESC'
        END
        --SET @strOppositeSort = ' ORDER BY ' +
        SET @strSortCriteria = ' ORDER BY ' + @strSort + ' '
    END
    ELSE
        SET @strSortCriteria = ''
    IF @strGroup IS NOT NULL AND @strGroup != ''
        SET @strGroupCriteria = ' GROUP BY ' + @strGroup + ' '
    ELSE
        SET @strGroupCriteria = ''
    --****    GET THE RECORDCOUNT           *****
    IF @strGroupCriteria != ''
    BEGIN
        SET @ParmDefinition = N'@internalCount int OUTPUT'
        SET @SQLString = 'SELECT @internalCount = COUNT(*) FROM (SELECT COUNT(*) FROM ' + @strTables + @strFilterCriteria + @strGroupCriteria + ') AS tbl (id)'
        
        exec sp_executesql @SQLString, @ParmDefinition, @internalCount = @totalRecordCount OUT
        IF (@@error!=0)
        BEGIN
            SET @errMessage = 'Error in dynamic sql clause - SELECT COUNT(*) AS SP_RECORDCOUNT FROM (SELECT COUNT(*) FROM ' + @strTables + @strFilterCriteria + @strGroupCriteria + ') AS tbl (id)'
            RAISERROR  20001 @errMessage
                RETURN(1)
        END
    END
    ELSE
    BEGIN
        SET @ParmDefinition = N'@internalCount int OUTPUT'
        SET @SQLString = 'SELECT @internalCount = COUNT(*) FROM ' + @strTables + @strFilterCriteria + @strGroupCriteria
        
        exec sp_executesql @SQLString, @ParmDefinition, @internalCount = @totalRecordCount OUT
        IF (@@error!=0)
        BEGIN
            SET @errMessage = 'Error in dynamic sql clause - SELECT COUNT(*) AS SP_RECORDCOUNT FROM ' + @strTables + @strFilterCriteria + @strGroupCriteria
                RAISERROR  20001 @errMessage
            RETURN(1)
        END
    END
    --******** NORMALIZE THE PAGING CRITERIA
    --if no meaningful inputs are provided, we can avoid paging and execute a more efficient query, so we will set a flag that will help with that (blnBringAllRecords)
    IF @intPageNo < 1
        SET @intPageNo = 1
    SET @strPageNo = CONVERT(VARCHAR(50), @intPageNo)
    
    IF @intPageSize IS NULL OR @intPageSize < 1 -- BRING ALL RECORDS, DON'T DO PAGING
        SET @blnBringAllRecords = 1
    ELSE
        BEGIN
            SET @blnBringAllRecords = 0
            SET @strPageSize = CONVERT(VARCHAR(50), @intPageSize)
            SET @strPageNo =  CONVERT(VARCHAR(50), @intPageNo)
            SET @strTopPage = CAST(@topPage as varchar(5))
            IF @totalRecordCount - (@intPageSize * @intPageNo ) < 0
            BEGIN
                SET @strPageSize = cast(@totalRecordCount - ((@intPageNo - 1) * @intPageSize) as varchar(50))
                --SET @strPageSize = cast(1 as varchar(5))
            END
        END
    --************************** NOW START DOING THE REAL WORK
    IF @blnBringAllRecords = 1 --ignore paging and run a simple select
    BEGIN
    
        EXEC (
            'SELECT ' + @strFields + ' FROM ' + @strTables + @strFilterCriteria + @strGroupCriteria + @strSortCriteria
        )
    
    END-- WE HAD TO BRING ALL RECORDS
    ELSE        --BRING ONLY A PARTICULAR PAGE
    BEGIN
        IF @intPageNo = 1 --in this case we can execute a more efficient query, with no subqueries
        BEGIN
            EXEC (
                'SELECT TOP ' + @strPageSize + ' ' + @strFields + ' FROM ' + @strTables + @strFilterCriteria + @strGroupCriteria + @strSortCriteria
            )
            IF (@@error!=0)
            BEGIN
                SET @errMessage = 'Error in dynamic sql clause - SELECT TOP ' + @strPageSize + ' ' + @strFields + ' FROM ' + @strTables + @strFilterCriteria + @strGroupCriteria + @strSortCriteria
                RAISERROR  20001 @errMessage
                    RETURN(1)
            END
        END
        ELSE --execute a structure of subqueries that brings the correct page
        BEGIN
            EXEC (
                'SELECT * FROM (SELECT TOP '+ @strPageSize +' * FROM (SELECT top ' + @strTopPage + ' ' + @strFields +' FROM '+ @strTables + @strFilterCriteria + @strGroupCriteria + @strSortCriteria + ') rev_tb ORDER BY '+ @oppositeSort1  + ') page ORDER BY '+ @oppositeSort2
            )
            --SELECT @strTopPage
            --SELECT 'SELECT * FROM (SELECT TOP '+ @strPageSize +' * FROM (SELECT top ' + @strTopPage + ' ' + @strFields +' FROM '+ @strTables + @strFilterCriteria + @strGroupCriteria + @strSortCriteria + ') rev_tb ORDER BY '+ @oppositeSort1  + ') page ORDER BY '+ @oppositeSort2
            IF (@@error!=0)
            BEGIN
                SET @errMessage = 'Error in dynamic sql clause - SELECT * FROM (SELECT TOP '+ @strPageSize +' * FROM (SELECT top ' + @strTopPage + ' ' + @strFields +' FROM '+ @strTables + @strFilterCriteria + @strGroupCriteria + @strSortCriteria + ') rev_tb ORDER BY '+ @oppositeSort1  + ') page ORDER BY '+ @oppositeSort2
                RAISERROR  20001 @errMessage
                    RETURN(1)
            END
        END
            
    END --WE HAD TO BRING A PARTICULAR PAGE
    
    --IF WE NEED TO RETURN THE RECORDCOUNT
    IF @blnGetRecordCount = 1
        SELECT @totalRecordCount AS SP_RECORDCOUNT
    
END
GO

SET ANSI_NULLS OFF
GO
SET QUOTED_IDENTIFIER OFF
GO


As you can see, it's a monster but it's pretty darn fast to run and allows you to pass in the tables, fields and sort order you need without having to rewrite the craziness for every new pagination query.

SQL Server 2005 makes this considerably easier with the new function
row_number() OVER (ORDER BY mycol)
which allows returns the current row number so you can grab the middle of the recordset as follows:
DECLARE @rowsPerPage int,
    @pageNum int,
 @startRow int,
 @endRow int

SET @rowsPerPage = 10
SET @pageNum = 3

SET @startRow = ((@pageNum- 1) * @rowsPerPage)+1
SET @endRow = @startRow + @rowsPerPage -1

SELECT * FROM (
 SELECT row_number() OVER (ORDER BY id) as resultNum, id
 FROM regTrackingHistory
) as numberResults
WHERE resultNum  BETWEEN @startRow AND @endRow

Apr 21
2006

SQL List of Numbers

Sometimes you want a sequence of numbers to join against, say a list of days in the month 1..31 and you want to have a row for each regardless of whether you have a matching row for that day, giving you a pivot table by day of the month.  Here is a SQL Server 2000 function to do this:

SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS OFF
GO

CREATE FUNCTION dbo.fn_getNumList (
/*
WRITTEN BY Adam Howitt adamhowitt@gmail.com
FUNCTION getDays
WRITTEN FOR SQL Server 2000
DATE 04.21.2006
Usage:
select currentNum
from fn_getNumList(0,60)
*/
@st_num int,
@num_to_add int )
RETURNS @numList TABLE ( currentNum int not null)
AS
BEGIN
    WHILE (@num_to_add > 0)
    BEGIN
        SET @st_num = 1+@st_num
        INSERT INTO @numList (currentNum)
        VALUES (@st_num)
        SET @num_to_add = @num_to_add - 1
    END
    RETURN
END

GO
SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS ON
GO

So for example, you can use the following to retrieve a count of orders for each day of the month, including those where there were no orders.

select t1.currentNum, ISNULL(t2.numOrders,0) as numberOfOrders
from fn_getNumList(0,60) t1
LEFT JOIN (SELECT datepart(month,create_dt) as dom, count(*) as numOrders
FROM order_tb
GROUP BY datepart(month,create_dt)) t2
ON t1.currentNum = t2.dom