Monday, September 26, 2011

Running totals in SQL Server

One of the most asked question of the age is the "running total problem" of SQL. When you have to generate a running total of a SQL Server table.

The Problem:
The problem which arrives is this that the user wants to calculate the running totals for a particular column in the database to be either used in a report or for reference to be stored in a column and used later on while querying the database.

Let us suppose the user has this kind of table....

Net Payment Due
Running Total
2011-08-19 12:23

2011-08-19 12:32

2011-09-14 15:03

2011-09-14 15:07

2011-09-17 11:14

2011-09-17 13:51

2011-09-19 10:27

The fourth column "RunningTotal" is what the user wants. He wants a running total of the corresponding payments defined over a period of time, so that he can easily see the Running Total of the Payments over a period of time.

The Solution:
There could be many solutions for this issue.
  • Modern day reporting tools give us the flexibility to calculate the running totals within the reports. This way the user won't have to calculate the running totals in the database.
  • Keep running totals in a Computed Column in a database and simply query that column whenever required.
  • Calculate the Running Totals in the same query whenever required.
First and the third scenarios are used when the user (in this case, the developer) only wants the running total to be calculated for a very few times and he would like to calculate the running totals only when is needed.

One should use the second case only when there is frequent need of the Running Total. The running total, thus would be stored in the database and the developer could access the running totals through a simple query of the column. As we all know, that computed columns have their drawbacks. I would strongly recommend to use the third solution.

The Options:
There are different options available for us to use when calculating running totals using SQL Server....

  1. Nested Sub Query
  2. Self Join
  3. Cursor
Let us see those options one by one...

1. Using Nested Sub Query:
We will use a sub query to calculate sub totals for a result set while fetching the result set in parallel. The query would be like this...

DataEntryDateTime ,
NetPaymentDue ,
NetPaymentDue + COALESCE((SELECT    SUM(NetPaymentDue)
 FROM      dbo.Payment b
), 0) AS RunningTotal
FROM    dbo.Payment a

Here, the key to the solution is the little 'WHERE' clause, which is restricting only the previous rows to be fetched from the database. 

Note: If your ID is not the primary key, You need to add an 'ORDER BY' at the end of the outer query to sort out the results as well.

2. Using Self Join:
In this method we will use a 'JOIN' statement to join to the same table and will produce the same results. Here is how...

a.NetPaymentDue ,
FROM    dbo.Payment a
INNER JOIN dbo.Payment b ON (b.ID <= a.ID)
GROUP BY a.ID , a.NetPaymentDue
ORDER BY a.ID , a.NetPaymentDue

If you want the 'Date' column to appear in the select list as well, you will have to put the above query in the sub-query and join to the same table one more time to do it...

3. Using Cursor:
The third option is to use the cursor, which (to me) seems to be the best possible solution for this kind of problem. This also proves the point that cursors are not always slow... :-)

NetPaymentDue MONEY ,
RunningTotal MONEY
@NetPaymentDue MONEY ,
@RunningTotal MONEY
SET @RunningTotal = 0
SELECT ID, NetPaymentDue
FROM dbo.Payment
OPEN rt_cursor
FETCH NEXT FROM rt_cursor INTO @ID, @NetPaymentDue
SET @RunningTotal = @RunningTotal + @NetPaymentDue
INSERT  @TempTbl
VALUES  (@ID, @NetPaymentDue, @RunningTotal)
FETCH NEXT FROM rt_cursor INTO @ID, @NetPaymentDue
CLOSE rt_cursor
DEALLOCATE rt_cursor
FROM    @TempTbl
Like the post.... Give it a Kick...... kick it on

Monday, September 5, 2011

Major reasons for software failure

There are often the cases, when the software is not shipped on time or even if it is shipped, it is not shipped with the quality that is expected of it, resulting in an eventual failure of the project.

A research conducted by market research firm Info-Tech Research Group says 95 per cent of information technology groups "are not delivering some number of projects on time or to the full satisfaction of the business executive."

While there may be many reasons for this, the major reasons involve the following:

Poorly defined scope:
In most of the cases, the reason is not the clarification of the requirements.

  • It may be because of an unreasonable deadline from sales/management
  • It may be due to overlooking the customer's expectations or it could have been preconceived notions from the start
  • It could have been a developer promising a particular UI widget (The "gee whiz" factor) or it could have been a developer not challenging the inclusion of a particular UI widget (which, the customer, wants desperately)
  • It could be that the projected savings were wrong or it could be that an important scope change was delayed until the next version
In the end, it all boils down to the user/customer expectations regarding a software. If you have met the customer's requirements, you have a successful software.

My friend, Hassan, has a very good post here, regarding the requirements gathering of a software project which can be a good reading source.

Unmotivated Developers:
In some of the cases, the software failure arrives because of a lack of interest or respect towards the software from some or whole of the development team. It may be due to the following reasons...
  • Development team is not motivated enough for the project
  • The hype against the project is not created before the arrival of the project to keep the team thinking about the project
  • Lack of interest from the authorities/management in the project sometimes result in the lack of interest from the development team
Lack of Domain Knowledge:
Sometimes, it is the case when developers are assigned tasks and they have not enough knowledge about the project or about the organization for which they are developing the project.
  • One of the reason could be that the development team has not been given much time to do R & D for the project
  • The development team hired for the project has not enough background experience for related developments
  • The development team hired for the project is incompetent for producing the goods on time
In the end, I think it all concludes to the fact that the customer satisfaction is the first and fore-most thing and all the above factors contribute to the satisfaction (or dissatisfaction) of the customer. 

As a project manager, we must look into these factors before going ahead for the development of the project.

We can minimize the risks mentioned above by doing the following simple tasks:
  1. Focus on features not implementation tasks.
  2. Work in iterations (like weekly or biweekly).
  3. Release working features to your staging environment in order of priority.
  4. Unit test your code as you go, so you're not slowed down by a bug-list that increases geometrically as you approach the release date.
  5. Be prepared to cut scope from the less important features. Stuff always takes longer than you think it will.
  6. Make sure you sketch out the UI in advance (if there is a UI), and show it to potential users.
  7. Test, test, and test some more. This seems counter-intuitive, but it saves more time than takes.