Disk Queue Length, incidentally, is something you can see in Windows 7+, under the Resource Monitor (Start > Task Manager > Performance tab > Resource Monitor > Disk tab)

Anyway, during large inserts, or imports, or selects, you may see your disk queue length spike up. This means that your computer is requesting data from the disk, but it’s busy spinning and getting data for different requests; so they start to rack up.

If you see a queue length of, say, 50, that means you have some heavy IO that’s sucking up all your disk availability and queueing up tons of requests — like when you’re restoring an 8GB database into SQL Server (please, use Attach next time.)

Two easy ways to increase performance in this case:

1. Not the Windows Partition: By default, SQL Server installs to C:\, the same place as Windows. Move the data files off of the Windows drive. You will immediately see an (often drastic) drop in queue length. Why? Because Windows makes a lot of requests, and if they start to queue up, it can make the queue much longer. If you have a second (physical, not logical) hard drive, that’s an ideal place to locate the files.
2. Split MDF and LDF Drives: Ideally, if you have three or more drives, you should put Windows on one, your MDFs on a second, and your LDFs on a third for maximum performance.

Again, unless you see the specific problem of disk queue length, you might not need to do all this. Plus, memory is usually a heavier bottleneck for SQL Server, not disk — so check that first.

How can you set the location of the MDF and LDF files? With new databases, when creating them, SQL Server shows you “Database Files” listed under “General,” with a “Path” property. Just change it to something else.

<appSettings>
  <add key="drive" value="C:\" />
</appsettings>

At runtime, you can access these values like so:

System.Configuration.ConfigurationManager.AppSettings["drive"]; // returns "C:\"

This is great for storing a few key/value pairs for different uses. But what if you wanted to store a list of strings? In our example, what if you wanted to store multiple drives? How can you do this?

One solution is to use multiple key/value pairs, like so:

<appSettings>
  <add key="drive1" value="C:\" />
  <add key="drive2" value="D:\" />
</appsettings>

Of course, this means you have to individually access the values at runtime; you can’t just grab them all into a single array or string.

The answer is surprisingly easy: just store the value as a delimited list and split it at runtime, like so:

<appSettings>
  <add key="drives" value="C:, D:" />
</appsettings>

And at runtime:

string[] drives = ConfigurationManager.AppSettings["drives"].Split(',').Select(s => s.Trim()).ToArray();

(The Select and Trim calls allow us to use white-spaces in the config file, and get a trimmed, whitespace-free list of drives.) This code would return, as expected, a string array:

drives[0] = "C:"
drives[1] = "D:"

Easy, right? But not so obvious. In fact, the obvious answer to this is to use a custom section. Personally, I feel that this is overkill for asimple list of strings, ints, or other basic types. (Technically, you can shove any serializable class in here; as long as you have a non-ambiguous delimiter, you’re good.)

Procedure or function "..." expects parameter '@foo', which was not supplied

This doesn’t make sense when you have code like:

string foo = getSomeFoo(); // May return null
SqlCommand cmd = new SqlCommand("SELECT * FROM someTable WHERE foo = @foo");
cmd.Parameters.AddWithValue("@foo", foo);
cmd.ExecuteNonQuery(); // Throws exception

You can see that @foo is, indeed, supplied. But in the case where the value is null, you will see an exception.

So what’s the solution? You can make sure that foo has a value, or some non-null value (like empty string or DBNull.Value), or you can conditionally add the parameter when needed.

And calling a stored procedure with a parameter that may be null? That, my friends, is a question for Google.

You can head over to the Random Number Generators page to download the latest source and binaries. (For those of you who don’t know, I’ve published source and binaries for generating random numbers in a normal distribution.)

Thanks to everybody who complained — if this library was not useful, I would never have gotten around to fixing it!

• Install NLog: Make sure you have NLog installed on the machine which you’re deploying to. (If you don’t, you should see your service crash, or errors in your Event Viewer.)
• Configure NLog Properly: Make sure that NLog actually works! You can do this by debugging into the service on your local machine (or via remote debugging). Make sure you have loggers that log both to console and to a log file, as you can verify NLog is working with the former.
• Copy NLog.Config to the .svc Directory: Some people mention that NLog.config should be placed where your .svc file is deployed. In addition to this, I find that you still need a copy in bin. For some reason, having only one log file (either one) didn’t work, but both together (identical files) worked.
• Check Your App Pool: Another issue I ran into is that IIS had my site configured using the classic pool; I actually had to change it to DefaultAppPool.
• Restart IIS: If you changed anything, restart IIS via the IIS console. It should work.

One more thing to mention: make sure you have some checks in place on the log size. Depending on your usage, your log files may quickly eat up gigs of space. You can gracefully handle this many ways, the easier being to archive logs every month and simply delete unnecessary ones. I use this configuration for rolling logs:

<targets>
  <target name="logfile" xsi:type="File" fileName="${basedir}/MyAwesomeService.log"
        archiveFileName="${basedir}/archives/log.{#}.txt"
        archiveEvery="Month"
        archiveNumbering="Rolling"
    maxArchiveFiles="6" />
  <target name="console" xsi:type="Console" />
</targets>

This limits my archives to six log files, rotated monthly; on the first of each month, NLog will generate a new file (exactly at midnight!) in /archives, with the filename log.n.txt (for n in {1, 2, …}).

And it should work! If it doesn’t, post a comment (especially if you found another piece of configuration or work that needs to be done before NLog cooperates).

Anyway, if you use views, you might be wondering how exactly they work, and how they affect performance. Are they cached? Are they more efficient than the equivalent SELECT statements?

As it turns out, the answer is “it depends.” SQL Server allows you to index views to optimize selection. (Indexing views has some caveats, and it depends on the underlying column types/aggregation.) At any rate, the implementation, according to this MSDN article, is:

• Indexed views are cached, and
• Non-indexed views are expanded as needed (into their constituent SELECT statement).

This means that if you use a view without an index, you can’t expect any performance gain (or loss) — it’s just a logical rearrangement of table data. On the other hand, adding any index will give you the benefit of caching. This means you can use an indexed view to replace an expensive query, because SQL Server caches the results. This is very useful if you have huge queries or jumbo, multi-table selects.

What’s more, the same MSDN page contains a veritable gem of a statement:

In SQL Server Enterprise, the SQL Server query optimizer automatically determines when an indexed view can be used for a given query execution. The view does not need to be referenced directly in the query for the optimizer to use it in the query execution plan. Therefore, existing applications can take advantage of the indexed views without any changes to the application itself; only the indexed views have to be created.

That’s a kicker of a statement. It means you can (secretly) optimize your SQL Server performance on the back-end by creating indexed views, without changing your application. This makes it easy to create/test/drop/repeat as necessary; no code changes necessary.

Simply amazing. I really suggest you read up on indexed views, and their caveats; they’re a very useful tool to have in your toolbox if you ever find a performance bottleneck in your database layer.

As a refresher, the assembly version resides in the file Properties\AssemblyInfo.cs. The file contains assembly-level declarations, which is to say, annotations for various assembly-level meta-data. (If you ever need to add any assembly-level annotations, like configuration for Eazfuscator, this is the place to do it.)

At the end of the file, you will notice a lovely comment block and two attributes:

// You can specify all the values or you can default the Build and Revision Numbers 
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]

The comment indicates how you can specify auto-versioning of the assembly version. You might think “easy, let me update my file like so:”

[assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyFileVersion("1.0.0.0")]

But there’s a gotcha here: you need to specify only the AssemblyVersion attribute for the auto-incrementing to work. With this configuration above, you will still see an assembly version of “1.0.*”, strangely enough.

The correct configuration is:

[assembly: AssemblyVersion("1.0.*")]

Save, build, and everything will work.

“Hey, that’s great,” you might be thinking. “Let me just update this through the project properties > Assembly Information button in Visual Studio.” So you blank out the File Version fields, only to be greeted with this:

That’s right — Visual Studio will only let you enter a blank File Version through the AssemblyInfo.cs file, not through the GUI! (Thankfully, if it’s already blank, and you edit some other properties, Visual Studio won’t complain.)

The Generated Number

You might be curious to know how Visual Studio generates the resulting numbers. In fact, you may wonder why it doesn’t start at, say, 1.0.1.1 and increment to 1.0.1.2, etc.

The actual details are specified on this MSDN page. The summary is that:

• The build number (the third number in the version) is equal to the number of days since January 1st, 2000. So if today is January 23rd, 2012, that’s approximately a value of 4400.
• The revision number (the final number in the version) is equal to the number of seconds since midnight, divided by two.So a build at midnight will result in 0, while a build at 1am will result in a value of 1800.

The end result is that every build results in a different version number (unless you run two builds at the very same second). The other caveat is that, if you have a build machine, you need to make sure your machine times are synchronized. Or at least be aware of the fact that a build machine in a different timezone may generate a build that seems newer (or older) than it should be.

Then you’ll see this (infamous) message:

Introducing FOREIGN KEY constraint 'FK_Foo' on table 'Blah' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.

This message can mean a couple of things; one of them is two ways a deletion can cascade down to the same table (i.e. delete table T, and there are two foreign key cascade deletions to table D). The other is that you’re trying to add foreign keys in a way that causes a cycle, i.e. it will cascade back to the same table.

Thankfully, Wayne Berry wrote a script to take care of this. It finds and prints out cyclical dependencies, like so:

dba.Foo -> dba.Bar -> dba.Foo

You can find the full script below.

SET NOCOUNT ON

-- WWB: Create a Temp Table Of All Relationship To Improve Overall Performance
CREATE TABLE #TableRelationships (FK_Schema nvarchar(max), FK_Table nvarchar(max),
    PK_Schema nvarchar(max), PK_Table nvarchar(max))

-- WWB: Create a List Of All Tables To Check
CREATE TABLE #TableList ([Schema] nvarchar(max), [Table] nvarchar(max))

-- WWB: Fill the Table List
INSERT INTO #TableList ([Table], [Schema])
SELECT TABLE_NAME, TABLE_SCHEMA
FROM INFORMATION_SCHEMA.TABLES 
WHERE Table_Type = 'BASE TABLE'

-- WWB: Fill the RelationShip Temp Table
INSERT INTO #TableRelationships(FK_Schema, FK_Table, PK_Schema, PK_Table)
SELECT
    FK.TABLE_SCHEMA,
    FK.TABLE_NAME,
    PK.TABLE_SCHEMA,
    PK.TABLE_NAME
FROM INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS C
      INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS FK ON 
        C.CONSTRAINT_NAME = FK.CONSTRAINT_NAME
      INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS PK ON 
        C.UNIQUE_CONSTRAINT_NAME = PK.CONSTRAINT_NAME
      INNER JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE CU ON 
        C.CONSTRAINT_NAME = CU.CONSTRAINT_NAME
      INNER JOIN (
            SELECT i1.TABLE_NAME, i2.COLUMN_NAME
            FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS i1
            INNER JOIN INFORMATION_SCHEMA.KEY_COLUMN_USAGE i2 ON
             i1.CONSTRAINT_NAME = i2.CONSTRAINT_NAME
            WHERE i1.CONSTRAINT_TYPE = 'PRIMARY KEY'
) PT ON PT.TABLE_NAME = PK.TABLE_NAME

CREATE TABLE #Stack([Schema] nvarchar(max), [Table] nvarchar(max))

GO

-- WWB: Drop SqlAzureRecursiveFind
IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = 
    OBJECT_ID(N'[dbo].[SqlAzureRecursiveFind]') AND type in (N'P', N'PC'))
DROP PROCEDURE [dbo].[SqlAzureRecursiveFind]

GO

-- WWB: Create a Stored Procedure that Recursively Calls Itself
CREATE PROC SqlAzureRecursiveFind
    @BaseSchmea nvarchar(max),
    @BaseTable nvarchar(max),
    @Schmea nvarchar(max),
    @Table nvarchar(max),
    @Fail nvarchar(max) OUTPUT
AS

    SET NOCOUNT ON
              
    -- WWB: Keep Track Of the Schema and Tables We Have Checked
    -- Prevents Looping          
    INSERT INTO #Stack([Schema],[Table]) VALUES (@Schmea, @Table)
    
    DECLARE @RelatedSchema nvarchar(max)
    DECLARE @RelatedTable nvarchar(max)
    
    -- WWB: Select all tables that the input table is dependent on
    DECLARE table_cursor CURSOR LOCAL  FOR
          SELECT PK_Schema, PK_Table
          FROM #TableRelationships
          WHERE FK_Schema = @Schmea AND FK_Table = @Table

    OPEN table_cursor;

    -- Perform the first fetch.
    FETCH NEXT FROM table_cursor INTO @RelatedSchema, @RelatedTable;

    -- Check @@FETCH_STATUS to see if there are any more rows to fetch.
    WHILE @@FETCH_STATUS = 0
    BEGIN
    
        -- WWB: If We have Recurred To Where We Start This
        -- Is a Circular Reference
        -- Begin failing out of the recursions
        IF (@BaseSchmea = @RelatedSchema AND 
                @BaseTable = @RelatedTable)
            BEGIN
                SET @Fail = @RelatedSchema + '.' + @RelatedTable
                RETURN
            END
        ELSE            
        BEGIN
        
            DECLARE @Count int
        
            -- WWB: Check to make sure that the dependencies are not in the stack
            -- If they are we don't need to go down this branch
            SELECT    @Count = COUNT(1)
            FROM    #Stack    
            WHERE    #Stack.[Schema] = @RelatedSchema AND 
                #Stack.[Table] = @RelatedTable
        
            IF (@Count=0) 
            BEGIN
                -- WWB: Recurse
                EXECUTE SqlAzureRecursiveFind @BaseSchmea, 
                    @BaseTable, 
                    @RelatedSchema, @RelatedTable, @Fail OUTPUT
                IF (LEN(@Fail) > 0)
                BEGIN
                    -- WWB: If the Call Fails, Build the Output Up
                    SET @Fail = @RelatedSchema + '.' + @RelatedTable 
                        + ' -> ' + @Fail
                    RETURN
                END
            END
       END
           
       -- This is executed as long as the previous fetch succeeds.
    FETCH NEXT FROM table_cursor INTO @RelatedSchema, @RelatedTable;
    END

    CLOSE table_cursor;
    DEALLOCATE table_cursor;    

GO    

SET NOCOUNT ON

DECLARE @Schema nvarchar(max)
DECLARE @Table nvarchar(max)
DECLARE @Fail nvarchar(max)

-- WWB: Loop Through All the Tables In the Database Checking Each One
DECLARE list_cursor CURSOR FOR
      SELECT [Schema], [Table]
      FROM #TableList

OPEN list_cursor;

-- Perform the first fetch.
FETCH NEXT FROM list_cursor INTO @Schema, @Table;

-- Check @@FETCH_STATUS to see if there are any more rows to fetch.
WHILE @@FETCH_STATUS = 0
BEGIN

    -- WWB: Clear the Stack (Don't you love Global Variables)
    DELETE #Stack
    
    -- WWB: Initialize the Input
    SET @Fail = ''

    -- WWB: Check the Table
    EXECUTE SqlAzureRecursiveFind @Schema, 
        @Table, @Schema,
         @Table, @Fail OUTPUT
    IF (LEN(@Fail) > 0)
    BEGIN
        -- WWB: Failed, Output
        SET @Fail = @Schema + '.' + @Table + ' -> ' + @Fail
        PRINT @Fail
    END
            
   -- This is executed as long as the previous fetch succeeds.
    FETCH NEXT FROM list_cursor INTO @Schema, @Table;
END

-- WWB: Clean Up
CLOSE list_cursor;
DEALLOCATE list_cursor;    
                    
DROP TABLE #TableRelationships
DROP TABLE #Stack
DROP TABLE #TableList
DROP PROC SqlAzureRecursiveFind

Pretty cool.

This is actually possible in ASP.NET MVC (as of MVC3), with a bit of work. You will need:

• Your ASP.NET MVC application (of course)
• MigratorDotNet (or your migrations tool of choice)
• Visual Studio Test Framework(or your testing tool of choice)
• A bit of practice and ingenuity!

Here’s how it works. Given that you already have your development environment working, along with your set of migrations to create your schema tables (including the core ASP.NET tables if you’re using the built-in login module), you just need to create a once-per-test-run method that will recreate the database and run the migrations.

Sounds complicated? It’s not. Here’s how it would look (with the tool stack listed above):

[AssemblyInitialize] // Run once per test run
public static void RecreateDatabase(TestContext context)
{
  DataContext dataContext = new DataContext(connectionString);

  // Drop all the foreign keys; they can prevent tables from being dropped
  dataContext.ExecuteCommand(@"DECLARE @sqlForeignKeys VARCHAR(MAX)

  SELECT @sqlForeignKeys = ISNULL(@sqlForeignKeys,'') +
  'ALTER TABLE dbo.[' + OBJECT_NAME(FK.parent_object_id) + '] DROP CONSTRAINT [' + FK.name + '];' + CHAR(10)
  FROM SYS.FOREIGN_KEYS FK

  EXEC(@sqlForeignKeys)");

  
  // Drop all tables
  dataContext.ExecuteCommand("EXEC sp_MSforeachtable @command1 = \"DROP TABLE ?\"");

  // Run migrations to rebuilt schema
  Assembly migrationsAssembly = Assembly.Load("..."); // main application assembly
  Migrator.Migrator m = new Migrator.Migrator("SqlServer", connectionString, migrationsAssembly);
  m.MigrateToLastVersion();
}

The code is pretty straight forward: we drop all the foreign keys first, then the tables. We use sp_MSforeachtable, which is a stored procedure that iterates over all tables. (We drop foreign keys first, because you can’t drop a table that has a foreign key pointing to one of its columns.)

Then, we get a hold of our main assembly (the one containing the migrations), and we run those.

And everything just works! The AssemblyInitialize attribute ensures that this code is only run once, when the test assembly is loaded.

And just like that, you have your very own Rails-style database being recreated on-the-fly for running tests. (Depending on your flavour of ORM, you can even add your own seed data here for testing.)

MVC3 is partially testable: you can create controllers and request views, even pull items out of the ViewBag. That’s great, but for many applications, it’s not enough; you want to be able to actually read the underlying HTML and check for the presence or absense of certain HTML content.

Thankfully, there is an answer. It’s not simple, but it is relatively straight-forward: deploy your debug directory of your web application onto IIS, and use a .NET conversion of HtmlUnit (a GUI-less browser written in Java) to run on IKVM, a .NET implementation of the JRE.

Credit for this amazing solution goes to Steve Sanderson, who documented the process (and built the binaries) on his blog. Essentially, you need to:

• Deploy your debug environment into IIS (giving you the benefit of seeing real IIS errors that don’t appear in Casini)
• Download the HtmlUnit and IKVM binaries
• Add all the appropriate references to your test project
• Compile, execute, and rejoice

Steve’s page does a great job of xplaining all the steps; just note that he has a link to the IKVM and HtmlUnit binaries at the end of his post (so you don’t need to build them yourself).

Once you have your app deployed, testing can be as simple as:

  [TestMethod]
  public void RequestingSecureUrlRedirectsToLogOn() {
    WebClient webClient = new WebClient();
    HtmlPage page = (HtmlPage)webClient.getPage("http://localhost/some/secure/action");
    WebResponse response = page.getWebResponse();
    Assert.IsTrue(response.getRequestUrl().toString().EndsWith("/Account/LogOn"));
  }

Run your tests (Visual Studio test framework in my case), and, tada! More complicated tests are possible (filling out forms, clicking buttons, checking response HTML), all though the HtmlPage API. For example, we could have extended that test to include:

((HtmlInput)page.getElementById("UserName")).setValueAttribute("bob");
((HtmlInput)page.getElementById("Password")).setValueAttribute("invalid");
HtmlPage resultsPage = (HtmlPage)loginPage.getElementById("submit").click();

string content = resultPage.asXml();
Assert.IsTrue(content.Contains("Login was unsuccessful."));

In this case, we filled in the UserName and Password fields, clicked Submit, and checked the HTML of the resulting page — that it contained an error message.

Pretty slick stuff. I highly recommend it if unit testing is something you prioritize. The potential for integration testing is pretty awesome.