Best method to implement a filtered search

Question

I would like to ask you, your opinion when it comes to implement a filtered search form. Let's imagine the following case:

• 1 Big table with lots of columns
• It might be important to say that this SQL Server

You need to implement a form to search data in this table, and in this form you'll have several check boxes that allow you to costumize this search.

Now my question here is which one of the following should be the best way to implement the search?

1. Create a stored procedure with a query inside. This stored procedure will check if the parameters are given by the application and in the case they are not given a wildcard will be putted in the query.

2. Create a dynamic query, that is built accordingly to what is given by the application.

I am asking this because I know that SQL Server creates an execution plan when the stored procedure is created, in order to optimize its performance, however by creating a dynamic query inside of the stored procedure will we sacrifice the optimization gained by the execution plan?

Please tell me what would be the best approach in your oppinion.

Answer 1

You might want to look at the answer to this similar question here: https://stackoverflow.com/questions/11329823/add-where-clauses-to-sql-dynamically-programmatically

We've found that a SPROC which takes in a bunch of optional parameters and implements the filter like this :

CREATE PROC MyProc (@optionalParam1 NVARCHAR(50)=NULL, @optionalParam2 INT=NULL)
AS 
...
SELECT field1, field2, ... FROM [Table]
WHERE 
  (@optionalParam1 IS NULL OR MyColumn1 = @optionalParam1)
  AND (@optionalParam2 IS NULL OR MyColumn2 = @optionalParam2)

will cache the first execution plan it is run with (e.g. @optionalParam1 = 'Hello World', @optionalParam2 = NULL) but then perform miserably if we pass it a different set of optional parameters (e.g. @optionalParam1 = NULL, @optionalParam2 = 42). (And obviously we want the performance of the cached plan, so WITH RECOMPILE is out)

The exception here is that if there is ALSO at least one MANDATORY filter on the query which is HIGHLY selective and properly indexed, in addition to the optional parameters, then the above PROC will perform fine.

However, if ALL the filters are optional, the rather awful truth is that parameterized dynamic sql actually performs better (unless you write N! different static PROCS for each permutation of optional parameters).

Dynamic SQL like the below will create and cache a different plan for each permutation of the Query parameters, but at least each plan will be 'tailored' to the specific query (it doesn't matter whether it is a PROC or Adhoc SQL - as long as they are parameterized queries, they will be cached)

So hence my preference for :

DECLARE @SQL NVARCHAR(MAX)        

-- Mandatory / Static part of the Query here
SET @SQL = N'SELECT * FROM [table] WHERE 1 = 1'

IF @OptionalParam1 IS NOT NULL        
    BEGIN        
        SET @SQL = @SQL + N' AND MyColumn1 = @optionalParam1'    
    END        

IF @OptionalParam2 IS NOT NULL        
    BEGIN        
        SET @SQL = @SQL + N' AND MyColumn2 = @optionalParam2'    
    END        

EXEC sp_executesql @SQL,        
    N'@optionalParam1 NVARCHAR(50), 
      @optionalParam2 INT'
    ,@optionalParam1 = @optionalParam1
    ,@optionalParam2 = @optionalParam2

etc. It doesn't matter if we pass in redundant parameters into sp_executesql - they are ignored. It is worth noting that ORM's like Linq2SQL and EF use parameterized dynamic sql in a similar way.

The awful 1 == 1 hack can also be avoided if you keep track of whether any predicates have yet been applied or not, and then conditionally apply the first AND only on the second and subsequent predicates. If there are no predicates at all, then WHERE disappears as well.

Note that despite the dynamic query, we are still parameterizing the filters, if present, so we have at least a first line of defence against SQL Injection attacks.

Answer 2

Start with whatever you think is easier to implement (I guess option 2). Then measure the performance for real world data. Only start optimizing when needed, not beforehand.

By the way, depending on how complex your search filters are, your task may not be easily solved without dynamic SQL. So even when you use a stored procedure, that most probably won't increase performance, as you already suspecting. On the other hand, if it helps, there are several types of hints (see http://www.simple-talk.com/sql/performance/controlling-execution-plans-with-hints/) you can add to an SQL query, dynamic or not, to help SQL server to optimize its execution plan.