Earlier today there was a thread on Twitter asking about what degrees and academic background people have who work on SQL Server. I volunteered to put together a reading list for those wanting to know more of the theory behind a relational database management system, rather than just how to use one.

Here I present a reading list that will take you from how to program well up to how to architect multi-threaded database servers. I've read all of these at some point between finishing my CS/EE degree in Edinburgh in 1994 and stopping dev work in 2005, and they're sitting on my bookshelf as I type this. They're all the best books I could find on the subject at the time, and they're all absolutely excellent. I've included Amazon.com links to the most up-to-date editions (because I'm nice like that ).

Programming

• The Art of Computer Programming, Volume 1: Fundamental Algorithms (Knuth, 672pp)
• The Art of Computer Programming, Volume 2: Seminumerical Algorithms (Knuth, 784pp)
• The Art of Computer Programming, Volume 3: Sorting and Searching (Knuth, 800pp)
• Algorithms in C++, Parts 1-4 (Sedgewick, 752pp)
• The C++ Programming Language: Special Edition (Stroustrup, 1030pp)
• The Design and Analysis of Computer Algorithms (Aho/Hopcroft/Ullman, 470pp)
• Design Patterns: Elements of Reusable Object-Oriented Software (Gamma/Helm/Johnson/Vlissides, 416pp)
• How to Solve It: Modern Heuristics (Michalewicz/Fogel, 554pp)

Underneath the RDBMS

• Computer Networks (Tanenbaum, 912pp)
• Modern Operating Systems (Tanenbaum, 1104pp)
• Distributed Systems: Principles and Paradigms (Tanenbaum, 704pp)
• Computer Architecture and Organization (Hayes, 624pp)
• Computer Architecture: A Quantative Approach (Hennessy/Patterson, 704pp)
• Cryptography and Network Security (Stallings, 744pp)

Concepts

• Parallel Processing and Parallel Algorithms: Theory and Computation (Roosta, 566pp)
• Decision Support Systems and Intelligent Systems (Turban/Aronson/Liang/Sharda, 850pp)

  Distributed Algorithms (Lynch, 904pp)
• Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery (Weikum/Vossen, 852pp)
• Transaction Processing: Concepts and Techniques (Gray/Reuter, 1070pp)

RDBMS architecture

• Database Management Systems (Ramakrishnan/Gehrke, 936pp)
• Introduction to Database Systems (Date, 1024pp)
• Database: Principles, Programming, and Performance (O'Neil/O'Neil, 870pp)
• Readings in Database Systems (Stonebraker/Hellerstein, 877pp)

You should also checkout the ACM Special Interest Group on Management of Data (SIGMOD), and the VLDB Conference - these are the premier academic conferences to do with database management systems.

This should keep you busy.. happy reading!

Over the last 18 months a group of SQL Server MVPs, led by Paul Nielsen and including Kimberly and I as editors, have been working on a book, responding to a challenge from Microsoft CEO Steve Balmer for MVPs to give something to the wider community than just our technical one. Now the book is available to purchase, with all profits (minus minimal expenses of the publishers) going to War Child International.

Buy the book, increase your knowledge, and help the children.

Rather than come up with my own blurb about the book's contents, I'm re-using the excellent write-up from the MVP blog, with their permission.  

With contributions from 53 SQL Server MVPs, SQL Server MVP Deep Dives, is organised in 5 parts: Design and Architecture, Development, Administration, Performance Tuning and Optimisation, and Business Intelligence. Within each part, you'll find a collection of concise and focused chapters that take on key topics like mobile data strategies, Dynamic Management Views, or query performance. The range of subjects covered is comprehensive, from database design tips to data profiling strategies for BI. The book features:

• Nearly 60 chapters focused on topics crucial for any SQL Server professional
• Edited by MVP Paul Nielsen, MVP Kalen Delaney, MVP Greg Low, MVP Adam Machanic, MVP Paul S. Randal, and MVP Kimberly L. Tripp
• Appropriate for SQL Server professionals of all levels
• Coverage of new features in SQL Server 2008

Whether you're just getting started with SQL Server or you're an old dog looking for a few new tricks, SQL Server MVP Deep Dives belongs on your bookshelf.

The authors of this book have generously donated 100% of their royalties to support War Child International. War Child International is a network of independent organizations, working across the world to help children affected by war. War Child was founded upon a fundamental goal: to advance the cause of peace through investing hope in the lives of children caught up in the horrors of war. War Child works in many different conflict areas around the world, helping hundreds of thousands of children every year. Visit www.warchild.org for more information.

You can purchase the book by clicking here.

The first public CTP of 2008 R2 was just released (see http://www.microsoft.com/sqlserver/2008/en/us/R2.aspx) and many people are rushing to try it out.

One word of caution: any database attached to a 2008 R2 server will have the database physical version number increased to 660. The database version number for 2008 RTM and SP1 is 655. This means that you won't be able to take that 2008 R2 database and attach it back (or backup/restore it back) to 2008 RTM or SP1.

I don't mean this as a reason to not play around with the CTP, just be careful you don't inadvertently upgrade a database that you still need to run on 2008 RTM or SP1.

For more info on database version number and how previous versions are not forward-version compatible, see these blog posts:

• Search Engine Q&A #13: Difference between database version and database compatibility level
• TechNet Magazine August 2008 SQL Q&A column which discusses physical version changes during upgrades
• Msg 602, Level 21, State 50, Line 1 on errors you'll see trying to attach or restore an uplevel database on a downlevel version

Hope this helps!

Several times this week I've been asked about how to become an MVP. A few people have posted on this, but here's my take. 

I think basically what it comes down to is that if you aspire to become something you're not already, you usually need to make changes in your life. Sometimes significant changes. But it's quite easy to prioritize the wrong thing by accident. Becoming an MVP doesn't mean compulsively answering every question on the SQL Server forums, or blogging huge amounts of stuff that others have already covered - but it doesn mean you have to get out there and interact with the community - it's all about the community. Here's what I recommend:

• Starting answering forum questions, but make *really* sure the answer is correct. Make sure you help out on the MSDN forums too, as these are one of the first places the MVP team looks, and aim to become a moderator if you can. This can really become an obsession - you have to ensure you're adding value rather than just answering for the sake of getting your score/reputation/ranking higher.
• Start a blog with regular interesting and informative posts about SQL Server (or whatever you're trying to become an MVP in). But again, don't be obsessive. It's easy to fall into the trap of posting stuff to try to get more people to read or become higher up a list. Check out this blog post I wrote: Are we too obsessed with rankings? And try to make your blog interesting with some entertaining stuff thrown in, for example Just how long should you make character fields? What's the longest word?
• Get on Twitter and join in the community. Get on Facebook or LinkedIn - somewhere that people can see something behind the professional persona. IMHO community isn't just about always providing answers - it's about knowing some of the people there too. Another blog post with more info: How Twitter and social networking changed my life...
• Start presenting. In the beginning this always sucks - it's hard to present when you've never done it before and most people find it terrifying. However, I'd say it's the #1 way to get involved in the community and get noticed. Don't aim for major conferences right away - it won't happen. Start with local user groups, SQL Saturdays, maybe do a podcast. In February I made my traditional Valentine's Day blog post to Kimberly a long description of how to speak in public, as a tribute to her helping me - see Public Speaking: A Primer.
• Find a mentor who's already an MVP to help you out. This is pretty crucial - you need someone who will tell you if you need to change the way you're doing something, maybe obsessing, maybe goals are a little off. I had Kimberly as my mentor, and she didn't hold back on the advice and constructive criticism.
• Be nice. No-one likes a smart-ass know-it-all who puts people down. This is a balancing act - don't be too humble either.
• Make friends with other MVPs. It's a community, right? Get out there and talk to them.

Lastly, take a step back and consider why you aspire to become an MVP. Do you just want the badge so you can show off? Is it something you want to tick off a list of achievements? Do you want to increase your professional standing? Do you think it will make you more attractive as an employee or consultant? Do you want to become a leader in the community?

Not all of these are valid reasons IMHO - I'll let you think which are and which aren't.

PS Great point from @sqlagentman - an MVP isn't something you're awarded, it's something you're formally recognized as already being.

Thanks to the attention of some scurrilous parasites, all comments will now be moderated. And to said parasites, please be aware that there's no point spamming my blog with comments linking to your websites as no-one will ever see them now. And I've contacted your hosting companies with your IP addresses, and a copy of each of your hundreds of spams.

Sorry folks - until the spam stops, comments are moderated. If it continues or gets worse, comments will be disabled.

Thanks

There's another DBA 'chain-blog' going around, this time started by Tim Ford, and I was tagged by Tom LaRock. The idea is that you're stuck on a desert island for six months with WiFi and you have to spend it doing something related to work. What would you spend the six months doing? If I break the chain then I won't get enough birthday cards to get in the Guinness Book of Records, or something else depressing, so I'd better join in.

At the moment, I'd probably say continue losing my life to Twitter, SQL forums, and blogging - but then I'd be blogging about blogging and there's the risk I'd get sucked into an infinitely-recursive blog post and I wouldn't get anything done. And Kimberly already thinks I blog too much, so I'd steer clear of that. I'd really prefer to spend the six months diving in the reefs around the island and checking out the stars in a sky with no light pollution, but I think it would be hard to make either scuba gear or a telescope from just coconut shells and sand, and I'm pretty certain that even if I could, the coconut-shell based air compressor for filling the scuba tanks wouldn't hold together when I powered it on either. But hey, where would the power come from? But if there's no power, how would the WiFi work? Hmm - the scenario's falling apart quickly - best cut to the chase.

I spent 15 minutes looking for a clean desert island joke to include here, then gave up, sat watching a heron fly past the deck and then figured out what I'd do. Now, you all know that I'm not a DBA, so I wouldn't have anything to do on a set of systems, but I do have a lot of things I'd like to do around SQL Server but have never had the time. Here are my top four:

Tool to explain corruption messages. The expert system of how to analyze all the corrution error messages that CHECKDB produces to find out what exactly it's telling you, and what may be deleted by repair, basically resides in my head. There's another copy that resides in Ryan Stonecipher's head (the guy that took over DBCC from me), but he's even less likely to have time to do this since he still works at MS. I'd love to program that expert system and make it available. Just like I'd like to put together the 'what exactly does each error message mean' PDF. I did it for 2000 and 2005 while at MS, and it took me 3 weeks. I have all that in my head too. I thought that writing the 2008 Internals book with Kalen would allow by in-head lazywriter to clean out some buffers, but it's all still there.

Write the book on maintenance and DR for involuntary DBAs. I keep getting asked what the good info is out there, and there's no one good place to go. The closest I've come to doing this is the first article I wrote for TechNet Magazine back in August 2008 - Top Tips for Effective Database Maintenance. There's a big need for a book aimed at non-SQL professionals as I see the same mistakes being made over and over in forums. This is another thing where I've got all the details in my head and spread through my blog and Kimberlys blog, but just need time to pull it together into a coherent story.

Write a script that will figure out the size of the next log backup. This is on my list of cool tools to write and give away on my blog. I've already done the how big will the next differential backup be, and the how much data will the next log backup include, but this one is a *lot* harder. I have a good idea how to do it, but I need a bunch of time to sit down and figure it all out in a solid script. And then test it lots.

Write a script that will produce a page checksum on every page. This has been on my list even longer. I know several ways to do this, but they all have undesirable side-effects, except one, which I haven't tried yet. This will be useful because once you enable page checksums after upgrading - nothing happens. A page doesn't get a page checksum until it's read into the buffer pool, altered, and then written out again. And there's no 'touch every page' tool.

Get hold of the SQL source code and add the following: online index rebuild of a partition, diff-based mirroring of FILESTREAM data, proper page-split monitoring, a new shrink algorithm that doesn't cause fragmentation, CHECKDB of a database within a backup without restoring the backup (my patent).

Ok - I guess I should tag some people too otherwise the chain will break and our house will be devoured by giant military squirrels... or something equally unlikely but tangibly scary. I choose my good friends Ward Pond, Greg Linwood, and Adam Machanic.

PS The title is a play on the BBC Radio 4 show 'Desert Island Discs'. It's supposed to be a geeky play on words. Well, I though it was funny and Tom had already used the 'Lovely Bunch of Coconuts' line. Thanks Tom.

Quick blog post to celebrate a bit of a milestone and round out the month of May - no blogging tomorrow.

Earlier today I was making a backup of all the content on my blog because I want to make sure I have a secure copy of it if multiple failures happen with site hoster etc. Look, come-on, this is me, I'm about the most paranoid person on the planet in terms of backups, corruption, HA, etc.

Anyway, as I was doing it, I thought it would be neat to count up the number of posts since I started this blog September 1st 2007 after leaving Microsoft the day before to join Kimberly. This is blog post #350 on this blog!

And as I was backing everything up in Word docs, one per month, I counted up the number of pages of 10-point font content on the blog. With the last few blogs posts, there's over 630 Word pages of content. Several book's worth. Which made me realize that I really really like writing and sharing information.

So - thanks for reading, commenting, cajoling, and asking questions. Keep it up and I'll keep it up too.

Vive la communite!

Last week's survey was another two-fold one - when you buy new servers, what architecture to you predominantly buy, and why?; when you buy new servers, which Edition of SQL Server do you predominantly buy, and why? Here are the results as of 5/17/2009.

For the first survey, the 'other' values were basically that 64-bit is purchased only if required. For the second survey, nine of the 'other' values were basically that Enterprise Edition is purchased only if required, one person uses Web Edition, and one person uses Developer Edition as they're just a developer.

I don't have a huge amount to say about these two surveys, I really just wanted to confirm my gut feel (and give you all a view of what's happening out there in the field).

For the architecture survey, I was entirely unsurprised to see that almost 80% of respondents are using 64-bit. It's widely known that 64-bit can give you better performance through the availability of more memory for SQL Server to use. Although you can use more than 4GB using AWE on 32-bit, that extra memory is only available for the buffer pool to use - not for general query processing - and AWE access does incur a little overhead. Much has been written about this and I'm not going to duplicate it here. A selection of articles can easily be found at http://www.google.com/search?hl=en&q=sql+server+64+bit+vs+32+bit&aq=1&oq=sql+server+64  More interesting is the small number of respondents who are not able to use 64-bit, either because their corporate policy is 32-bit (maybe because of cost?), they have software that doesn't run on 64-bit systems, or specifically because 64-bit is too expensive.

A few people only use 64-bit if required, and (I imagine) prefer to save money if it not by sticking with 32-bit. I wonder how long it will be until 32-bit servers are not available at all? Certainly future versions of Microsoft server software are starting to become 64-bit only - for instance Exchange 2010 (see here) and SharePoint Server 2010 (see here), will the next version of SQL Server be 64-bit only?

For the Edition survey, again, unsurprising results. 55% use Enterprise Edition for one or a combination of the various "-abilities" (if you make performancability a new word), with almost 10% more having the option to use it if required, and 16% having to stay with Standard because of budgetary constraints. What you may be surprised to see (but I wasn't from my time working side-by-side with the SQL marketing team) is that 20% of respondents don't need Enterprise Edition. There were a lot of improvements to the database engine in SQL Server 2000 and SQL Server 2005 that meant that for intermediate workloads (hmm - just made that up, but you know what I'm trying to say), Enterprise Edition isn't needed. And with synchronous database mirroring available in Standard Edition, you can implement a great, low-cost high-availability plan without paying for Enterprise. As you can see, I don't get any kickbacks from the SQL team for selling Enterprise Edition  Saying that, however, there are a lot of very cool features in Enterprise Edition in the various versions - but if you don't *need* them, why pay for the higher Edition just for the sake of it?

One thing I will say to summarize, it's important to look at your requirements in terms of performance, availability, etc to make the right choice of server architecture and SQL Server Edition. If you make the wrong choice, do you think your company will pay to rectify your mistake before the next round of capital-expenditure? Probably not - but you'll definitely pay for the wrong choice with several years worth of hassle trying to make the system be more performant and easily recoverable than your choices allow.

As always, thanks for participating in the surveys - I've had a bunch of mail from people who like to see what other people are doing.

Next up - this week's survey!

Go to www.wordle.net/create and enter a URL. Here's what it thinks of my current blog homepage:

Cool!

A sad, tear-jerky story about how a socially-inept, stay-at-home shut-in, SQL expert gets a new lease of life and finds he really does have friends out in the big world after all. LOL. Not. I'm definitely not socially-inept - managed to persuade Kimberly to marry me, right? Well, that was really down to the classic pick-up line "Hey baby, wanna hear about some undocumented DBCC commands?"

Ok, enough nonsense. I wanted to make some comments about Twitter and Facebook and how they're actually really useful to me.

I must admit, I was a vociferous opponent of Facebook until the start of April. Kimberly caved first under peer-pressure, then I ridiculed her for several days before succumbing to temptation myself. I've heard Facebook called the 'adult MySpace', and I have to agree. Why did I take the plunge? First off I was curious, and went through the phase of trying to read everything and post contrived funny stuff, but then sanity took over (and I had to do some work) and I realized that Facebook is a very useful tool for staying social while not physically seeing anyone.

As anyone who knows me knows, I *hate* telephones with huge passion. I mean, I really really dislike talking on the phone. I only turn on the ringer on my cell phone when Kimberly and I are apart, and that's not very often. Even when I'm abroad, I'll use Skype to talk to people. Really, phones are evil. They interrupt your life when someone *else* wants to talk to you. There's a reason we have a PBX in the house - so that it answers the phone and sends us email with the voicemail in (very handy for travelling as much as we do too). If I ever have to talk on the phone I like to schedule it. Now all this is quite weird, because in person I'm chatty and sociable. I just hate phones.

I digress. My point with the phone rant? I don't stay in touch with people over the phone, and most of the people I know are spread around the world, and socializing over email just sucks and isn't practical either. How do you keep up with hundreds of people on email? Most people struggle to get through just the work email they get every day, especially all the tech-folks that I'm friends with. Facebook solves that problem. None of the people I'm friends with on Facebook are the kind that have verbal diarrhea and update their status every time they sneeze or drink a glass of water. However, people post photos of places they've been, their kids, gardens; links to their blog posts, and other people's interesting blog posts; and generally stuff that their friends might find interesting. And the best thing is that it's like a blog - I don't have to read it if I don't want to, and nothing clutters up my inbox. And I post stuff they might find interesting. People I know only through work get to know the non-work side of me better without all the tedious small talk. It's like getting to know someone from a distance. Kind of weird I suppose. Best of all, I don't have to talk on the phone.

Now, Twitter. When I got on Facebook I saw all these status updates with #s and @s and thought what the hell is all that nonsense? Why would people type to each other with a 140 character limit? No way I'm getting involved in that. Then some people started badgering me to get on it (I-won't-name-names-Jason Massie-you-know-who-you-are, oops), and others warned me my productive life would end and we'd end up living in a trailer park eating baked-beans. So, of course my curious, addictive personality got the better of me and I joined. And yet we're still here in our house in Redmond and I still don't like baked-beans.

After the initial spend-all-waking-moments-reading-everything phase, things have settled down and I now believe that Twitter is very, very useful. It's a different crowd from Facebook (although there's some obvious cross-over) - and it's kind of like instant messenger with lurkers. I hate instant messenger. Just as bad as phones. As soon as Kimberly or I log in to IM, we're inundated with people wanting to talk to us. So we don't - ever. We actually have private IM names that no-one knows apart from us that we use when one of us is travelling alone (not since last December) just so we can IM without people butting in.

I digress again. So why is Twitter useful to me? Everyone on Twitter (in my circle) does something to do with SQL Server - so it's a kind of SQL-late-night-party-line (but not the kind you see advertized if you ever turn on TV at 3am). I'm able to keep in touch with other MVPs, people I know from conferences, and people I've never met - and learn about problems, ask questions, answer questions, and generally be active and social in the community - more than I would be able to do without Twitter. Without Twitter, I'd be limited to forums, classes, and conferences - none of which make for really being long-term *social* in the community, just active in it. And, to be honest, there's the real work aspect of it too. If more people know who I am, more people are likely to want to attend a conference session, workshop, or class that I'm teaching (the trick though, is that I have to be interesting or funny or both - not marketing crap). The more people I know in the SQL world, the more I find out about issues that I can use for blog posts and in magazine articles. It works both ways, as Kimberly has her blog title - "Improving my SQL skills through your questions!" - people get lots of info from me, I get info from them, and we all have a nice, happy, shiny community.

So yes, I was a serious detracter of both of these tools, but now I'm a serious proponent of them. If you use them the right way, not trying to read everything the 500 people you're following say, or tweet every time your dog barks or your wife is sick (like I did today), then they can be really useful. And I think people like to see something more personal than "conference Paul".

Brent Ozar (who has the distinction of being the first person I followed) has some great posts on Twitter on his blog and everything he says is true. I won't follow someone unless it looks like they're saying interesting/funny things. I don't care how many people follow me and I don't try to post stuff to get more people to follow me - I have plenty. If you don't like what I post, don't follow me, I'm not changing.

This has been a bit of a ramble, but I wanted to get a bunch of stuff off my chest. Has Twitter and social networking really changed my life? Honestly? Yes. In a big way? No, but in a way I wasn't expecting.

It's actually made me more social.

PS If you too want to risk living in a trailer and eating baked-beans, I'm on Facebook (search for paul@sqlskills.com) and Twitter at http://Twitter.com/paulrandal. Kimberly's also there on Facebook (search for kimberly@sqlskills.com) and Twitter at http://Twitter.com/kimberlyltripp.

5/11/09: Little bit of a rewrite today as it seems some people are taking what I'm saying the wrong way.

Over on Ed Bott's blog, last week he showed some inside info about the notorious "SQL Server problem" that caused the Windows 7 RC downloads to be so slow. The reason given was that SQL Server fragmentation caused the server to spike CPU and an index rebuild fixed the problem. Then the SQL CAT team posted that the problem was (paraphrasing) that the download team had only planned for a 100% jump in traffic, and got a 500% jump. The server (me: for a so-far undisclosed reason) couldn't handle the load, so a new server was dropped in and (me: this is interesting) operational practices were changed. Today (5/11/09) I find out that there's an ongoing investigation into just why the server couldn't handle the load. Notice I'm not saying the SQL Server was deficient in any way, but that the server as a whole couldn't handle the load.

Based on the initial information, and the CAT team not stating what the actual problem was, plus they had to change an operational practice (e.g. more frequent index rebuilds to remove fragmentation), we (Kimberly and I) came up with a hypothesis as to what might have happened. Remember that saying that the server wasn't setup for a 500% jump in load could mean a very large number of things. It could mean that the raw number of inserts overloaded the hardware on the server. It could mean that the application logic didn't scale to the extra workload. It could mean that the database schema didn't scale to the extra workload. The latter is the one that we're hypothesizing.

Whatever the actual technical issue was, there are two real problems: 1) the lack of planning, whether it was in terms of hardware capacity, application logic, schema design - whatever 2) the way the problem was handled by the MS PR folks, which not only made out that it was a SQL problem, but made people like me think that of-course-it-wasn't-a-SQL-problem, but it must have been a design/schema problem.

I discussed this with Kimberly and here's what we think happened (from her initial idea). There's been some discussion on how simply downloading a package could cause fragmentation - it's just a SELECT right? Wrong. If you look at what happens when you download, there's a GUID (Globally Unique IDentifier) that gets generated for your download, and there must be a table that tracks downloads, so the GUID is entered into a table. The table most likely has a clustered index, with the random GUID value as the primary key (or the high-order key in a composite key).

Every download thus produces an insert into the clustered index at a random location in the index. As the index pages get full, they eventually need to split, so more inserts can occur in the index at points defined by the random key. These 'page splits' are very expensive, causing lots of IO, log record generation, and fragmentation (both logical fragmentation that interrupts range scan performance and reducing page density from pages becoming less full). All of this takes CPU, which explains some of the spiked CPU from all the downloads. If there are queries that are scanning this data too, the fragmentation would cause terrible performance issues, resulting in more, smaller IOs - and also adding to the CPU load. Rebuilding the index (as was explained in the original blog post) would remove the existing fragmentation, increasing performance again, but wouldn't remove the page-split issue - which is why (apparently) they're going to rebuild that index every night now.

(This is the 100-level explanation for non-SQL geeks, a more in-depth explanation can be found on Kimberly's blog at GUIDs as PRIMARY KEYs and/or the clustering key.)

One of the comments below from Andrew Kelly suggests it could also be an ad-hoc plan problem - again, not a SQL Server issue, but a problem with the application. That could equally be explained by unplanned workload blowing capacity.

So was this a SQL Server issue? No. SQL Server did exactly as it was told, and has no choice where in the index to insert the new records - the GUIDs provide the insertion point. The problem was in the developer who created that schema without testing it with a very high workload - and the lack of a database maintenance plan to pro-actively find and resolve fragmentation issues, etc, etc. 

Now, this is pure conjecture, based on the facts that have come out so far. However, nothing that's been said since the issue (by the SQL team or anyone else) has made me question the hypothesis that some design/app issue led to the problem. As I said, capacity planning can mean a huge number of things - but saying that it's a capacity issue certainly doesn't mean it's NOT a schema design/app logic issue.

Once we hear what the root cause of the capacity overload problem was, I'll post more details. In the meantime, of course I'm not making out that it's a SQL Server problem - if you tell SQL Server to (for instance) use random GUIDs a the clustered, primary key - under high load it's not going to perform well. Inherent SQL Server problem? No. Human problem? Yes.

I hope this rewritten version is clearer now.

I was in a discussion earlier today, and it's one I've had lots of times in the past - both inside and outside Microsoft and the SQL team: why doesn't SQL Server do automatic <defrag, index creation, refactoring, de/normalization, backups, CHECKDBs, etc>?

Some of these were considered while I was still on the team, and I was *very* cautious about them. Here's why, taking automatic table de/normalization as an example (this morning's discussion). Assuming the logic to do the de/normalization based on usage patterns can be worked out, here's why I don't think it will ever happen (just off the top of my head):

• If it's wrong even once, no-one will ever use it again.
• When should SQL Server change the table definitions? What time of day is best?
• Where should the new table be placed? In which filegroup?
• What if there's no space to do it - should the database autogrow because of an automatic process? And then should it shrink afterwards?
• How far should the change plan parallelize?
• Who manages the transaction log size if the table is really large? Automatically take log backups? What if there's no space for the log backups? What about space on the log shipping secondaries? And the time required to roll-forward the log on the log-shipping secondaries?
• What about the impact of extra transaction log on database mirroring? Both in terms of the huge SEND queue preventing log truncation, and the huge REDO queue preventing the mirror coming online quickly?
• What if a cluster failover occurs during the operation and the rollback prevents the database coming online within the company's RTO?
• What if there are an replication topologies setup? How do they get quiesced for the schema change?
• How does the app get changed to handle the different schema, or do views get created automatically? Indexed views or not?
• What if Change Data Capture is running? Should it automatically create a new capture instance? What if both are already in use?
• How do SPs get updated to the new schema? Build in the datadude engine as part of SQL Server?

My point: it's a lot harder than you think to just put some automatic behavior into SQL Server. 

Now, saying that, I hope that some of the others do happen at some point, but with lots of config parameters so I can control them.

After being mercilessly pressurized by many people (and I blame Jason Massie the most), I've joined Twitter and I'll be commenting lots on it. Why did I finally cave? Well, I've been seeing lots of little things in forums and newsgroups that don't merit a full blog post but are worth letting people know about. So - feel free to follow along. My username is 'PaulRandal' - original, huh? I'm sure I'll get just as sucked into Twitter as I have to FaceBook - luckily I can type fast :-)

Enjoy!

PS You might not find me in Twitter search until I've posted a few times - try http://search.twitter.com/

This feels like a bit of a strange post for me to make, but I want to make it anyway, and don't take it the wrong way...

I've recently started following SQLBatman's blog and I was astounded that he had to post a follow-up to his list of SQL blogger rankings (disclaimer: including ours - thanks!) because he'd been getting emails from people who obviously weren't as high in his list as they thought they should be, even though they were *his* rankings of their blogs. It struck me that with the amazingly connected society we live in now, and the ubiquitousness of things like blogs, Facebook (to which we're unfortunately addicted), and Twitter (which we're sternly resisting so far) - maybe we're becoming a little too focused on how popular our blogs are or how many people are following our Twitter feeds?

I used to be like that. When I first started blogging about 3 years ago on the Storage Engine blog at Microsoft, and then when I left and switched to this one a year later, I was pretty obsessed with how many people were reading the blog posts. Was I posting interesting info? Could I get away with jokes? Would my opinions offend someone? Was anyone linking to me? Why aren't I in XYZ's blog-roll?

Then I realized that, to be honest, the obsessing was taking the fun out of blogging and now I don't pay that much attention to the stats and just post stuff I think is interesting. I wanted to do this blog post to give a piece of advice to new bloggers and those who thought they should be higher up in, or just in, SQLBatman's (or anyone else's) rankings: don't get wrapped up in the numbers and rankings, and don't complain when someone doesn't rank your blog as highly as you think they should, or list it in their blog-roll. It's very easy to get bitter when a post doesn't get the numbers you think, or no-one links to it, and then start thinking along the lines of "why am I spending all this time doing this when no-one's watching?"

People are watching - there are thousands of 'lurkers' our there who you'll never see and never hear from, but they're watching and learning from what you post. Just focus on the content and if it's good you'll be discovered and/or ranked higher. Don't think about what to post that will make your ranking higher, think about what to post that's unique, interesting. and useful to people in the community. You can't really control what people think of what you post. I'm sure some people won't like this post, but I'm going to post it anyway.

From Field of Dreams 2: "If you build it, they will come".

Not an April Fool (although SQL Server Central has a great one) - Kimberly and I were both re-awarded MVP status for another year - cool! Don't expect any blog posts this week from either of us - we're on vacation with the kids down on St. Pete beach in Florida. Back to work next week. Have a great week!

Just noticed this come up on my Facebook news feed - how to code assignment statements when programming (not in T-SQL):

1. if (foo == 4) then ...
2. if (4 == foo) then ...

#1 is the most natural way to write the conditional expression, but has a *massive* potential for introducing bugs that are very, very hard to find. If the second '=' is omitted, the conditional expression suddenly becomes an assignment statement and evaluates to true. Nasty.

#2 is the best way to avoid such problems, even though it seems unnatural. If the second '=' is omitted, it becomes an assignment statement but the compiler will barf, because you can't assign a value to a constant. (Some languages and data types may be immune to this - so this isn't a blanket statement - it's certainly a problem in C++.)

#1 was the cause of several bugs that took me a while to find while working on the Storage Engine code - all C++. I always use #2.

PS if (foo = 4) won't compile in C# if foo is an int, but you can come up with weird cases that do compile and break. Who uses managed code anyway?!? ;-) (ok, no hate mail please)

I'm very pleased, and deeply honored, to announce that I've been made a Microsoft Regional Director. This is one of a very small group of people (about 120 worldwide) who Microsoft sees as influencers of, and liaisons between, the Microsoft community at large. Apart from me, the other SQL MVPs who are also RDs are Kimberly and Greg Low.

I looked around for a good description of what RDs do and what the RD program is all about and found two blog posts:

• Microsoft Regional Directors don't work for Microsoft
• Who are the Microsoft Regional Directors?

And you can get the complete list of RDs (I'm not listed there yet) at the RD site The Region.

Time for a few drinks tonight!

There's a long-running discussion with people tagging each other to post advice for people new to SQL Server, about what they know now and wished they'd known ealier in their lives/careers - lot's of SQL MVPs and other luminaries have been doing it and I've been tagged now by my good friend Ward Pond - here's his entry.

The idea is to give two pieces of advice to help people out. My two came to mind almost instantly.

The first is a tenet I live by - there's no fate but what you make. It's actually a quote from the Terminator 2 movie and it basically means that nothing happens to you unless you make it, and you're responsible for your own life. This applies equally to life and to your career.

If you're not in an optimal place in your life for whatever reason (happiness, job, city, partner), then it's up to you to change it. And you should have the confidence to try. Sometimes you might try and fail, but at least you can say to yourself that you've tried. I've changed jobs, cities, and partners a few times each and (luckily for me) it always worked out. Sometimes the change was hard to make, sometimes it wasn't. But I knew it was up to me if I wanted a change so I had no choice but to make it happen or adapt to the current situation.

For your career, and this is where I'm touching on SQL Server, there are some situations that you may get into that are entirely of your own creation. Someone breaks into your server because you didn't check security. The company goes under because the sales database was lost and you didn't provision backups. You get a bonus because you tuned the indexing strategy so performance could handle the big sales weekend. Whatever. I believe that everything that happens to you is from your own making (apart from 'random' things like car accidents). You may say that something happened in your job that you had no control over - but you took that job... It's an interesting way to look at life, but that's what I know now.

The second thing is purely about SQL Server. If you're a DBA, some day in your career you will encounter database corruption and you will be responsible for clearing it up. Don't stick your head in the sand and fool yourself that it won't happen to you. It will. Be prepared. Take backups. Check your backups. Come up with an HA strategy. Practice restoring. And so on. I've been involved in hundreds, maybe more than a thousand, of corruption cases while at Microsoft and beyond. The over-riding take-away for me from that (ongoing) experience? People are unprepared and don't know what to do. That's what I know now.

I hope these are helpful and I've fulfilled my obligation.

I hereby tag: Greg Linwood, Jason Massie, and Bob Beauchemin.

PS Kimberly was tagged - her post is here.

There's been an interesting discussion on SQLServerCentral about whether this question is valid for a DBA interview: what's the name of the executable that runs SQL Server?

My view is that it's a perfectly valid question, based on the cliched premise that the more you know, the further you go. I've conducted hundreds of interviews for positions at DEC and Microsoft and I've always been more impressed with people that knew more stuff (and why that stuff was useful to know - rather than just knowing weird facts by rote) than people who said they could look it up. To me, if someone knows things like the name of the SQL Server executable, it says to me that they've had more experience dealing with interesting situations (like having to start the server in single-user mode, or divide up resources on a multi-instance server using WSRM). And for that, they'd go higher up my 'hire' list than someone who would have to look it up. Of course, that's just one out of a large number of traits and characteristics that I'd be looking for.

So, my question to you is, do you think that's a good interview question (coupled with something to weed out those who don't know why it would be useful)? Vote in the survey below, and reply with any questions you think really are invalid for a DBA interview.

Thanks

PS And don't worry, it's not all going to be surveys from now on - I'm just having fun with a new toy. Some good internals posts coming up next week! 

One of the opinions that was expressed yesterday by our developer friends (a bunch of Regional Directors - and look, Kimberly's the featured RD today on http://theregion.com/) is that SQL Server is just plumbing. Consider the developers (or app architects) as plumbers. They see SQL Server as a few pieces of copper pipe joining a few parts of the app together, with maybe some flow control in there somewhere, like below. (You might need to right-click and select 'show picture' - I don't know why - I just do databases :-))

There shouldn't be anything in a piece of simple copper pipe that impedes the water flow, right? That's what you'd think. The problem is that mental model of a simplistic data store is wildly inaccurate. What they think of as a bunch of simple copper pipe and some taps is a really a very complicated machine, like below.

There should really be another step in the software development lifecycle - during the design phase, there should be a question that says 'are you doing anything with SQL Server? (Y/N)'. If the answer is 'no', the lifecycle model continues. If the answer is 'yes', then STOP and consult a DBA or database-savvy developer. Just like working on a house, most people will call a plumber rather than mess around with pipe-work themselves.

Kimberly's going to pick this topic up big-time on her blog - keep watching there!

Today we've spent a lot of the day in discussions with some folks about developers vs. DBAs, and how it's often the case that the two don't work together. Developers need to know the effect of their design choices on the database, and DBAs need to educate the developers. There should be a close working partnership between the two but bad communication and ignorance often get in the way - the DBAs don't know what the app devs want from them, and the app devs don't understand *why* the DBAs are so protective of the data tier. Neither side understands the day-to-day challenges of the other. Misunderstanding breeds conflict breeds animosity breeds intransigence and uncooperativeness. One of the eye-opening things sometimes is teaching DBA stuff to developers and vice-versa so they see the challenges and technology choice implications both ways.

We were pointed to this spoof video about a DBA-dev confrontation - http://www.benhblog.com/2009/03/linq-dba-vs-developer.html - pretty funny.

If you're new to the blog then you may not have seen my Search Engine Q&A series (or seen it and not realized what it is). It started out that I was watching the incoming search engine queries that hit the blog and worked out what common questions people were searching for, but then I turned it into a 'what are common problems on the forums that I see'. Anyway, there are a bunch of questions that I've answered that make some interesting reading and touch on some pretty diverse topics. Here they are in chronological order, with links. Don't forget that I fastidiously categorize all posts and you can use the category links on the left hand side of the blog.

1. Running out of transaction log space - how to tell why your log is growing and two common cases - no log backups and a long-running transaction
2. How to move a database while database mirroring is running
3. Mirroring - how long does it really take to detect a failure with database mirroring and how to combine mirroring and clustering
4. EMERGENCY mode - how to access a RECOVERY_PENDING or SUSPECT database
5. Multi-core vs. single-core CPU performance
6. Using the undocumented fn_dblog log analysis tool - how to see whether a transaction is contained in a backup
7. Multi-file backups - how to restore from a backup file containing multiple backups
8. How index maintenance can break mirroring
9. Updating constraints - how to update the constraints in the partitioning sliding-window scenario
10. Combining index and statistics maintenance
11. Designing a global-ready database
12. Multi-file databases to avoid contention - for tempdb and for user databases - should you? And the follow-on about the dangers of sweeping generalizations
13. Difference between database version and database compatibility level
14. How 3rd-party drivers can break DBCC CHECKDB
15. Using mirrored backups
16. Concurrent log and full backups
17. Backing up a VLDB - whether to split the VLDB into filegroups or smaller databases
18. Determining the uptime of SQL Server
19. Myths and misconceptions around index rebuilds
20. Database 'boot page' and boot page corruption
21. 'File header' pages and file header page corruption
22. Which pages cannot be single-page restored?
23. How to recover from a full transaction log
24. Why the transaction log can't be instant initialized
25. Why log backups aren't always the same size as the log itself
26. Myths around what can and cannot cause corruption
27. How does the storage engine find variable-length columns?

 Enjoy!

As you may have been seeing over the last few weeks, there are intermittent errors in some parts of mine and Kimberly's blogs. Our hosting site and our blog guy have figured out that there's an ASP version mismatch which is causing the problems - and they're going to fix it this week.

So - apologies for the problems - they've been bugging us too. If you hit an error, refreshing a few times should clear it.

[Edit: 03/08/09 - looks like everything's working now!]

It's become something of a tradition for me to do a Valentine's Day blog post (the 2008 one about SQL Server blogs is at Wow - so many blogs!) as we don't do anything special otherwise - I don't need a "Hallmark Holiday" to tell Kimberly I love her .

Ahem - well this year I'm choosing a topic where I owe a lot of my success to Kimberly - public speaking. Kimberly is renowned as a world-class presenter on SQL Server and I flatter myself that my reputation is getting up there too - with a *lot* of help from Kimberly over the years. In a recent blog post over on SQLServerCentral.com, Andy Warren kicked off a lively debate on speaker skills and how to improve - a general sentiment was that new speakers are often afraid to ask for help from more seasoned speakers, so in this post I'm going to brain-dump as much advice as I can for all the upcoming speakers out there.

Public speaking is something of an art and a science - there's no totally right or wrong way to do it and you have to find the style that works. One of the top speaker trainers that Microsoft uses (and a friend of ours), Richard Klees, says that slides should have a minimal number of words and you should speak slowly. If you've ever seen Kimberly or I present, you'll know that we do the opposite - speak fast and have packed slides - but it works for us. Since my first terrifying conference session at TechEd US in June 2006, I've presented countless chalk-talks, sessions, workshops, and multi-day classes around the world and I'm continually honing my skills. In no particular order, here are a bunch of things to consider that I've learned along the way - I'm going to be very honest with my thoughts, I'll probably forget some stuff, and I may wander a little...

• Choosing a subject. This may seem pretty obvious, but only speak about things you know. I'd never try to do a session on using the CLR, or writing stored procs - even though I know a few things, I can't speak authoritatively and confidently about them. The best sessions are those on subjects that you know really well. Don't try to cover too much in a single session - have a balance of depth and breadth. Be aware of the level (depth) of your session and only go to the level that you know about the subject. Different conferences use different levels, but here's one I put together for the MVPs working on the MVP book, based on DBCC:
  • 100 - Beginner (e.g. what does 'corruption' mean?)
  • 200 - Intermediate (e.g. what do I do when corruption is detected?)
  • 300 - Advanced (e.g. how do I do take advantage of partial database availability and online piecemeal restore?)
  •  400 - Master (e.g. how can I fix broken system tables using the DAC and server single-user mode?)
  • 500 - SQL Server Internals (e.g. how does the read-ahead in DBCC CHECKDB differ from regular adaptive range-scan read-ahead?)
• Know the audience. This plays into choosing a subject - I wouldn't present on corruption survival techniques to a bunch of hard-core developers, nor would I present the internals of CHECKDB to a group of involuntary DBAs. Make sure that what you're talking about and how you talk about it is appropriate to the audience otherwise they'll feel frustrated and things won't go well.
• Demos. People like to see things in action - so most technical presentations include demos. A good presenter is equally comfortable talking off slides or talking through a demo. Demos need to be appropriate but are hard to get just right. People need to be able to follow what you're doing - if you're flying between windows and never explaining what you're doing, that's not really a demo. If you have a demo where something takes a minute to run, fill the gap by explaining about what's happening. Protracted silence is not good in a presentation. Make sure your demos work - broken demos will happen to everyone (the joke is that there are 'demo gods' and some days they just don't smile on you) but that should not be the norm. Make sure your demos are predictable and you know how to recover if something goes wrong. Run through your demos on the machine you're going to do them on - I remember writing a corruption recovery demo and being startled in the live presentation when a database I'd corrupted at home behaved differently on SQL 2008 than SQL 2005. Don't have over-complicated demos - they will break. Most importantly - a demo should show something you've talked about in action. If not, why are you doing it?
• Nervousness. Everyone gets nervous - no matter how many times you present. The only thing that improves is how well you deal with it and how quickly it goes away. Well, I guess you also get less nervous as time goes on, but there's always that little buzz of concern whenever I'm going to present. I think that's a good thing though - if you get too complacent, you won't do as good a job of preparing and making sure everything goes smoothly. Even for my corruption decks, I still spend 5 minutes skipping through the slides before the session to remind myself of the order of things and to get comfortable that I haven't forgotten that I'm doing a particular demo, for instance. Once you start speaking, the nervousness should go away pretty soon as you get into the swing of things. One trick I taught myself when I was in China in 2006 presenting other people's decks was to practice the first two minutes of the session until I knew what I was going to say off by heart. By the time I'd gotten through the prepared opening, I was comfortable and in the zone. Mental preparedness is a big part of not being nervous - I call it being in the zone - you've got your head ordered with what you're going to say, your demos are all prepped, you've got everything setup and you're just waiting for the green light. Kimberly and I both have our little routines that we go through to get in the zone.
• Putting together a slide deck. This is the meat of the whole thing - your slides are what will guide you through your presentation. They need to tell a story, with a beginning, a middle, and an end and then need to flow well rather than jumping around between topics. I like to have the end of one slide lead into the next one. Writing good slides takes time to learn  - I'm constantly tweaking slide decks to make them better. There's lots of stuff written about writing slides so I won't go into much detail - the number one thing I'd say is to use a readable font and project in 1024x768. Don't pack your slides full (even though we do sometimes) and don't write down everything you're going to say. A slide should really summarize a couple of minutes of speaking. Don't feel like you have to say everything that's on the slides. Make sure that someone reading the slides offline can also get the idea of what you want to say.
• Powerpoint. Much has been written about what to do and not to do when using Powerpoint. I could spend a whole other blog post on it, but this one's long enough. Just go watch this funny 4-minute video - http://www.youtube.com/watch?v=cagxPlVqrtM (there are a ton of videos out there - use "bad public speaking videos" in Google - but the link I gave is one of the best-known). Enough said. One thing I will say though is the old "a picture speaks a thousand words" - I could spend 5 minutes describing a data record structure to a room full of people and none of them will have the whole picture in their head. Instead I have a slide with a picture of a record structure on it with annotations. Oh yes, and animations - they're great but they don't work very well on a printed copy of the slides...
• Using humor. Making the audience laugh is a good way to get yourself and them audience relaxed (yes, the audience needs to relax too so they can absorb what you're saying). I don't like to tell jokes when I present, but I can usually recall appropriate Dilbert cartoons - and when Kimberly and I co-present, we make fun of each other, which most people find makes the atmosphere really genial.
• Co-presenting. Only ever do this if you're *very* comfortable with your co-presenter, otherwise it can be disastrous. Kimberly and I will only ever co-present with each other now (although Kimberly's done it before with a few others) as we're fine with interrupting each other and complement each other well. However, we had some issues when we first started doing that and we had to learn to tone down the "cuteness" of being co-presenters and husband-and-wife - and we have to be careful with what we say to each other depending on the feel of the audience - jokes about being her being blonde, or me being Scottish might not go over well. Co-presenters need to be matched in terms of ability, style, confidence, etc.
• Why are you there? Give the audience some indication of why *you* are the one standing up there presenting to them. At the start of each slide-deck I present, I have a bio slide, but I only point out the relevant stuff. In fact, this last year in the corruption sessions I've taken to saying "here's a whole slide of stuff about me, but all that's really important today is that I wrote CHECKDB and repair, so that's what we're going to talk about". The audience can read all the other stuff later if they want - the odds are they already know about you anyway.
• Self-promotion. Be very careful. By all means have a slide with some of your qualifications, books, etc on it, but don't labor the point. People aren't there to hear about how wonderful you are. Don't blatantly plug your services, books, etc within your slides. We got dinged by someone at TechEd US in 2008 because they thought when we mentioned our blogs we were trying to get people to visit our website to sell them stuff. We now make a point of saying that our website has no advertizing on it - just free info and all the blog links in the slides are more free info. On the other hand, public speaking is a good way for you to advertize yourself and become well-known. The golden rule here is that if you present really well and come over as authoritative on the subject, people will naturally remember you and contact you if they're interested in working with you.
• Projection. You need to make sure the audience can hear you. That doesn't mean you have to shout, but your normal speaking voice is usually not loud enough to carry across a room. Don't talk down to the floor and if in doubt, use a microphone. At any conference, you *will* be forced to wear a wireless mic so try one out if you get the chance. Make sure you speak clearly and have good pronunciation. This is especially important if you're presenting in a country where the language you speak is not the first-language of the audience (e.g. when I present at TechEds outside the US, or if you're presenting in English but that's not your first language - quite common in the US).
• Tech check. Make sure that your laptop is compatible with the presenting equipment. Nothing says 'amateur' more than turning up to your session 5 minutes before it's due to start and finding that your laptop doesn't display properly. On the other hand - beware of over-zealous technicians that can really interrupt you getting 'into the zone' before your session starts. However, *always* be nice to the technicians otherwise you'll get a bad reputation at conferences - I've heard of this happening. Make sure you're really familiar with your laptop and how it displays.
• Dress code. This depends on where you're presenting. At TechEd, it's brown or khaki slacks plus a TechEd speaker shirt. When I teach anywhere else, even user groups, I wear jeans, shoes and a dress-shirt. Dressing smartly makes me feel more professional and comfortable. Really the bottom line here is that you should be appropriately dressed - a 3-piece suit for a user-group is over-kill, but be careful not to under-dress too.
• Practice. As with most things, the number one way to get better with public speaking is to practice. There's a good reason why conference organizers (ourselves included) want to know your speaking experience - proof that you've had lots of practice and won't tank in front of an audience. Try giving your presentation to an empty room, or to a friend or colleague. Best thing is to have a mentor that'll help you.
• Getting a mentor. Try to find someone who is a good speaker and that you trust. Have them watch your presentations and give you frank feedback. You'll improve in leaps and bounds by having someone point out where you're going wrong.
• Feedback. If someone's nice enough to give you constructive feedback on your presentation technique, take it as such. Don't bristle or be offended - they're trying to help you. Be open to criticism (self- or from others) - it's the only way you'll get better. If you're giving feedback, think about who the person is that you're giving feedback to and how they're going to take what you have to say. Managing teams of people from diverse cultures at Microsoft helped me immensely here - feedback is hard to give and receive.
• Take-aways. While you're writing your presentation, think about what you want the audience to take home with them. I like to have a slide at the end that spells out what I think the take-aways are. Don't ever end on a negative point though otherwise that's what people will remember the most. You want to make sure that people don't leave the room wondering what the message of your presentation was.
• Writing an abstract. You need to make sure that there's enough info in the abstract to make it compelling for people to want to attend the presentation - or not. Nothing annoys an audience more than an abstract that gives a false impression of what's in your presentation - that means that people are wasting time watching something they weren't expecting. The title of your presentation also has to reflect the content for the same reasons. Here's one I've used for the last year that I think is a good example:
  • Corruption Survival Techniques
  • Your database is corrupt - what do you do? Well, it depends! How critical is the data? Do you know what's really wrong with the database? What does all that DBCC CHECKDB output mean? Should you restore or repair? It’s all about limiting downtime and data-loss when a corruption occurs - from knowing the tools to understanding the implications of choices you make. In this demo-heavy session Paul and Kimberly will give you insight into how to recover from corruption without making things worse. Most importantly you'll get step-by-step instructions for dealing with the more common scenarios.
• Tangents. Don't be afraid to take tangents, but make sure it doesn't screw up your timing and that you get back on-topic. I keep a LIFO stack in my head to make sure I get back to where I started and will often say out-loud "Popping the stack back to...". It shows that you're flexible and able to wander away from the main stream of the presentation but that you care that the audience gets to hear everything you were orginally planning to say.
• Whiteboarding. Once you're really comfortable presenting, you should be able to switch back and forth between using your slides, and doing ad-hoc stuff with a whiteboard - or even doing a whole chalk-talk without any slides. Whiteboarding takes some getting used to - write legibly and big enough so everyone can see. Be aware of where you're standing in front of the whiteboard and who can't see because you're blocking them - so move around. Write in black or blue - red and green can't be seen very well from a distance. For extensive whiteboarding, the marker will start to run dry because of the angle you're holding it at - so make sure you have a few spares. We love whiteboarding - and in fact our first "date" was two hours whiteboarding partitioning schemes right after we met for the first time at TechEd. Sweet memories... 
• Props. I find that I'm much more relaxed if I have something in my hand to play with - like a paper clip or a water bottle cap - something small that won't distract the audience.
• Clickers. If you're serious about presenting, buy a wireless clicker so you don't have to walk back to your laptop to change every slide - it looks really bad and says that you're an amateur. The one Kimberly and I use is at LaserMouse.com. Make sure you have spare batteries with you too - but don't go too extreme - we travel with 5 laptops between us - long story - see here...
• Movement. This really comes down to personal preference - but I don't like to stand in one place. If you walk around a bit it gives the audience's eyes something to follow and can be more engaging that just standing still. But don't wildly gesticulate too much - both Kimberly and I do this sometimes when we get carried away. If you have a podium, don't get into the habit of hiding behind it and using it as a barrier between yourself and the audience - come out from behind it and get used to feeling exposed. You'll get used to it and will become a much more confident speaker. If you don't, odds are that one day you won't have a podium and then you'll feel and look uncomfortable.
• Dealing with questions. This is another personal preference and depends on your comfort level with what you're presenting - questions can be very off-putting. No-one likes to be caught out but if you are, never make something up or try to weasel out of answering. A really powerful statement is to say "I don't know the answer, but I'll find out - come up and see me at the end". People like to see that you're confident enough to admit you don't know everything. Don't fall into the trap of letting people's questions take over the session - know when its time to move on and politely say something like "we need to move on to get everything covered but I'll stick around for questions at the end". Don't take too long answering a single question, and don't let someone waffle on with their question either. Oh, most importantly - repeat the question for the audience - either explicitly, or say that you'll repeat the question as part of the answer. Think about what questions people might ask and try to forestall them by answering them as part of the presentation - sometimes I'll have a a Frequently Asked Questions slide that I'll run through so everyone gets answers.
• Timing. It's really important that you don't run over your allotted time. The audience needs a break, the next speaker needs to setup - it's just impolite. On the flip-side, you don't want to finish 15 minutes early in a one-hour session otherwise people think they've been short-changed. Getting the timing right is very difficult to do when you're first starting out - you need to take into account demos, questions, problems but above all else, you need to keep track of the time while you're presenting. Don't look at your watch every five minutes though - have an easily readable clock on the presenting desk or better yet, a clock on the wall at the back of the room.
• Eye contact. As you're speaking, move you eyes around the room and look at people - it makes a connection and shows you're confident. It also lets you see people's reactions to what you're saying. This is hard to do at first but if you force yourself it'll become second-nature pretty quickly. Don't go too fast though and you don't have to hit everyone in the room. In a large auditorium with lights shining in your face, it's impossible to see the audience, so just go through the motions as if you can see them anyway - it'll look much more natural. 
• Thinking of a word. My brain has this weird thing that it does - every so often I'll be in mid-speech and the multi-syllable word I really want to use next just won't come. This can be really off-putting and worrying the first time it happens but I've learned to recognize when I'm not going to be able to think of the word and to describe my way around it, sometimes even saying "I can't think of the right word". I know this happens to other people too - just don't spend 20 seconds standing there trying to rack your brain when it does - accept it as sign of fallibility in your aging brain and move on.
• Hesitation. There's a natural tendency for you to stop every so often while you're brain processes what to say next. At these points, the best practice is that no sound comes out of your mouth. You want to avoid repeated sounds like "erm", "ok", "so...". It's incredibly hard to stop yourself doing this but if you can vastly reduce how often you do it, your presentation will look a lot more polished and you'll seem more confident. My commute used to be about an hour each way into Microsoft, and I trained myself to not make noise when hesitating by spending a week trying to give a 5 minute speech on CHECKDB while driving without saying "erm". It took a while and you'd be surprised how difficult it can be to work up the courage to start giving a speech to yourself in the car. Recently I was using Camtasia to record about 8 hours of presentations for Microsoft and I had the same problem getting started - mouse pointer hovering over the Start Recording button - you just have to go for it.
• Confidence. Passion. Enthusiasm. If you can exude these three things when you present then you'll go a long way. Nobody wants to sit and watch someone who doesn't sound like they believe what they're saying, or drones on in a monotone voice. When it comes down to it, what Kimberly and I present is really dry, technical information - how to recover from corruption, how to tune your indexing strategy, etc - but we're *really* into what we're talking about and that shows in how we present. If you're not interested in what you're presenting, you're going to find it hard to present it with any energy and convince your audience that what you're saying is worthwhile. A good way to feel confident is to tell yourself that you know more about your topic than most people in the room and that's why they're hear to listen to you - but don't get freaked out if you see someone in the room who you *know* knows more than you. They're still there because they're interested in what you have to say.
• Honesty. There's no better way to bring the audience onto your side than to be honest with them about some feature or behavior that doesn't work. Don't try to give the marketing spin on a feature or avoid the difficult question because the answer isn't what people want to hear. But on the other hand, don't be downright negative either. Present a balanced view and give an objective opinion. If you think what you're saying is going to disappoint people (like whenever I present about the new setup method for failover clustering in SQL 2008), add something funny like "I know, it's pretty sucky, but I'm just the messenger!"
• Arrogance. In no way do you want to have the audience think that you're better than them - it really turns people off to be talked down-to. In some of my earlier sessions I would sometimes come across as arrogant when I was trying to express incredulity and you have to try hard to say things the right way. For instance, in TechEd US 2007 I presented a talk on database corruption and was listing some of the worst things that people do to corrupt databases. When I mentioned restarting SQL Server, I said something like (paraphrasing) "wow, what do these people know that I don't? Restarting SQL Server isn't going to magically fix the corruption". Yuk. Although I was trying to express disbelief, that first sentence just screamed arrogance, and I got dinged for it in some people's session ratings and comments (and Kimberly shouted at me!). Now I'm very careful to always word things to avoid coming across the wrong way.
• Empathy. One of the best ways to ingratiate yourself with the audience is to empathize with them. Make the audience feel like you've been in the same situations they have, you've had to deal with the same problems they're seeing, you've felt the same frustrations with SQL Server that they do. Your talk should let them know you feel their pain and that you're going to help them with some of the info you'll present. If the audience feels like you're speaking from experience they're more likely to want to hear what you have to say. But be careful not to take this too far or it can feel contrived and dishonest.
• Total no-nos. There are some cardinal sins which almost guarantee you'll irritate and annoy people (at best) or get thrown out of a conference (at worst - I've seen this happen at TechEd). In today's age of (almost extreme) political correctness, you need to be careful that you don't overstep any lines of good behavior or start talking about inflammatory topics - but don't be scared to be a little controversial. I'll happily call bad design decisions in SQL Server "daft", and I did while still at Microsoft too, but I'd never use words like "stupid" or "idiot". Don't tell dirty jokes. Don't ever discuss politics - you're guaranteed to annoy at least one person in the room and you're not there to discuss that. Same goes for anything to do with nationalism, sexism, racism, parochialism, xenophobia, etc - these are all very, very thin ice and you'll get into trouble quickly. Never use a four-letter word while you're presenting - you may not find them offensive but many people do. In fact don't swear at all - if you can't express yourself while presenting without resorting to swearing for emphasis then you shouldn't be presenting. All these things can make people feel very uncomfortable when they hear them in a session. One cliche to consider is - would you be happy playing a recording of your session to your grandmother?

As you've no doubt gathered from much of the above, public speaking involves a lot of psychology and it can be really fun tweaking sessions repeatedly to see how the audience reaction changes. I really hope that all of this will prove helpful to some of you and I'd love to hear about experiences and things you've learned that I may have missed. This turned out *much* longer than I thought (a 5000-word essay), but luckily I'm badly jet-lagged from being in Asia for 3 weeks and I woke up at 3.30am.

To round out I want to leave you with a quote from one of my top-5 favorite, utterly stunning movies - Gladiator. In one scene, the old ex-gladiator Proximo (played by the late, master orator Oliver Reed) is telling the fallen Roman general Maximus (played by the excellent Russell Crowe) how to succeed: "Listen to me. Learn from me. I was not the best because I killed quickly. I was the best because the crowd loved me. Win the crowd and you will win your freedom." Now, I'm not trying to say with this that public speaking is like gladiatorial combat, but that success comes solely from pleasing the crowd. If your talk ends with the audience thinking they've just been entertained, learned something useful, and not wasted their time then you've succeeded and can feel profoundly satisfied as the applause rings around the room.

And with that it just remains for me to wish my wonderful wife Kimberly Happy Valentine's Day!

Thanks for your patience while our blog engine back-end was updated. You should find that the response time is much better now. You'll notice that all the blog post URLs have changed - we have a conversion layer in place so all prior blog posts will be seamlessly accessible through their new and old URLs. There are a few kinks being ironed out still - the Recent Posts pane on the left will be replaced with an On This Page pane, so that Category-based browsing (which I really need for classes!) works again.

If you see anything that looks broken, please let me know.

Thanks!

Thanks!

Ok - this post is a little strange and fun. I was thinking about word length and how it relates to designing software/schemas to support multiple-languages. How far do you have to go in your research to figure out the maximum string length to support? So I started digging about and found some interesting things about words. Here are some examples.

• If you're putting together a schema to support hospital patient records, you might have a field for disease name. In that case, you'd have to allow for pnuemonoultramicroscopicsilicovolcanoconiosis which has 45 letters (caused by breathing in siliceous volcanic dust). A field for surgical procedure would have to support hepaticocholangiocholecystenterostomies which has 37 letters (creating a connection between the gall bladder and the hepatic duct). What about a field for how a measurement was obtained - electroencephalographically with 27 letters (using an electroencephalograph).
• A schema to support chemical names could really be unlimited given the nature of systematic names for chemicals. The longest one in the dictionary is an acid called tetramethyldiaminobenzhydrylphosphinous with 39 letters (and given a few minutes I could probably draw its chemical structure by following the systematic method I learned at school :-)). The longest published chemical name is a kind of tobacco mosaic virus - ACETYLACETYL-SERYL-TYROSYL-SERYL-ISO-LEUCYL-THREONYL-SERYL-PROLYL-SERYL-GLUTAMINYL-PHENYL-ALANYL-VALYL-PHENYL-ALANYL-LEUCYL-SERYL-SERYL-VALYL-TRYPTOPHYL-ALANYL-ASPARTYL-PROLYL-ISOLEUCYL-GLUTAMYL-LEUCYL-LEUCYL-ASPARAGINYL-VALYL-CYSTEINYL-THREONYL-SERYL-SERYL-LEUCYL-GLYCYL-ASPARAGINYL-GLUTAMINYL-PHENYL-ALANYL-GLUTAMINYL-THREONYL-GLUTAMINYL-GLUTAMINYL-ALANYL-ARGINYL-THREONYL-THREONYL-GLUTAMINYL-VALYL-GLUTAMINYL-GLUTAMINYL-PHENYL-ALANYL-SERYL-GLUTAMINYL-VALYL-TRYPTOPHYL-LYSYL-PROLYL-PHENYL-ALANYL-PROLYL-GLUTAMINYL-SERYL-THREONYL-VALYL-ARGINYL-PHENYL-ALANYL-PROLYL-GLYCYL-ASPARTYL-VALYL-TYROSYL-LYSYL-VALYL-TYROSYL-ARGINYL-TYROSYL-ASPARAGINYL-ALANYL-VALYL-LEUCYL-ASPARTYL-PROLYL-LEUCYL-ISOLEUCYL-THREONYL-ALANYL-LEUCYL-LEUCYL-GLYCYL-THREONYL-PHENYL-ALANYL-ASPARTYL-THREONYL-ARGINYL-ASPARAGINYL-ARGINYL-ISOLEUCYL-ISOLEUCYL-GLUTAMYL-VALYL-GLUTAMYL-ASPARAGINYL-GLUTAMINYL-GLUTAMINYL-SERYL-PROLYL-THREONYL-THREONYL-ALANYL-GLUTAMYL-THREONYL-LEUCYL-ASPARTYL-ALANYL-THREONYL-ARGINYL-ARGINYL-VALYL-ASPARTYL-ASPARTYL-ALANYL-THREONYL-VALYL-ALANYL-ISOLEUCYL-ARGINYL-SERYL-ALANYL-ASPARAGINYL-ISOLEUCYL-ASPARAGINYL-LEUCYL-VALYL-ASPARAGINYL-GLUTAMYL-LEUCYL-VALYL-ARGINYL-GLYCYL-THREONYL-GLYCYL-LEUCYL-TYROSYL-ASPARAGINYL-GLUTAMINYL-ASPARAGINYL-THREONYL-PHENYL-ALANYL-GLUTAMYL-SERYL-METHIONYL-SERYL-GLYCYL-LEUCYL-VALYL-TRYPTOPHYL-THREONYL-SERYL-ALANYL-PROLYL-ALANYL-SERINE - with 1185 letters.
• Probably the one that's going to catch most people out is place names. The bank Kimberly and I use won't allow a town/city name of more than 30 characters. That's fine for the USA, where the longest place name has 24 letters (Winchester-on-the-Severn in Maryland or Washington-on-the-Brazos in Texas). However, if the back-end database is coded to only support 30 characters, that wouldn't work around the world:
  • In Wales, there are two longest names are Llanfairpwllgwyngyllgogerychwyrndrobwyllllantysiliogogogoch with 58 letters and Gorsafawddachaidraigodanheddogleddolonpenrhynareurdraethceredigion wth 66 letters.
  • In New Zealand, there's a hill called Taumatawhakatangihangakoauauotamateaturipukakapikimaungahoronukupokaiwhenuakitanatahu - 85 letters and that name used to be in general use.

Pretty interesting - or as my kids like to say supercalafragalisticexpialidocious! (34 letters :-))

I'd be interested to hear of longest words in other languages apart from English - please leave a comment. Thanks

Well, now I'm no longer a 'Softie'. It feels a little strange after having been on the SQL team for 8.5 years but I'm really jazzed about all the stuff we'll be doing over the next year and more here at SQLskills. We're taking a small vacation before diving in - first thing I'll be doing when I return is revamping and updating all of my content from the SQL Server Storage Engine blog and reposting the updates here. After that I'll be posting a bunch on Katmai and working through my backlog of blog topics. I can't wait to get started - stay tuned...

Cheers