The answer is actually quite simple thanks to a little promoted feature of SSMS. Follow the steps below to see how

1. Right click the database you wish to export data from
2. Select ‘Tasks’ then ‘Export Data’
3. You will see theSQL Server Import and Export Wizard. Click ‘Next’ to go past the welcome screen and make sure your source is correctly selected (it will have been if you started by right clicking the database), then click ‘Next’ again
4. Now you can choose the destination database, note that for Azure you need to select ‘Net Framework Data Provider for SqlServer’ in the destination drop down
5. This will cause connection properties to appear. For Azure you need to set the data source,userid, password andinitial catalog. I would also recommend setting connection encryption as I have below.
6. You can now either copy data directly from a table or view or write your own query. Such power! Here I have elected to copy data from a table in the source database, but bear in mind that you could easily write a query to limit the number of rows or transform the data in some way.
7. Choose the table or tables with data you want to copy. The wizard will automatically try to choose a matching table on your destination database or suggest auto creating it if it does not exist. Click ‘Edit Mappings’ for each table you are transferring to see where your data will go.
   Here I have chosen two tables that exist in both source and destination and have the same structure and a third table that is different on the destination database. By highlighting that row and clicking ‘Edit Mappings’ you can choose exactly where your data will go on a row by row basis. Again, the wizard will auto suggest the best fits.
   Here there are no columns that are named the same so the wizard leaves it entirely up to you. You can also choose to preview the data.
8. Having mapped your source to your destination, click ‘Next’. Thefinal step is to either run the package immediately or save it as a package toSSIS. I usually run immediately if it is just a quick transfer I am after. Here I have selected both.
9. Finally you can click ‘Finish’ to execute the package. Note that you have created an SSIS package and you will see feedback on the progress as it goes, including number of rows transferred. If it encounters any problems it will stop and tell you.

Overall an excellent way to quickly transfer data from one system to another. Especially useful where you need to transform that data, map between two different schemas or send it to Azure.

What are you waiting for? Go Play!

Presuming you have already designed your tables and they are present in your project;

• Create a folder in your project called StaticData then add a sql script to generate the data you require, one for each table.  For instance I have a date dimension table called Dim_Date and have created a script called Populate_Dim_Date.sql.
  
• Next create a script called ‘Script.PostDeployment.sql’ in the root of your project by right clicking your project and choosing ‘Add’ then ‘Script’
  
  and choosing ‘user scripts’ in the SQL Server section, followed by ‘Post-Deployment Script’, then clicking the Add button.
  

This will create a script that runs following the full deployment of your project and you can direct it to step through each of your sql creation scripts in the StaticData folder we created earlier. To do this you simply add a SQLCMD read command to the post deployment script for each of the sql files you want to execute. Here I have added a command to read (include) the script I created for the date dimension.

Note that to prevent an infestation of the red squigglies you will need to select the icon that represents placing the editor in SQLCMD Mode, in my image above it is the icon with a blue border.

Now when you publish your project your tables will be populated according to your scripts – easy peasy!

If the database you are referring to is also a part of the solution – for instance if you are creating a data warehouse and you have a supporting staging database, it is very similar to the process of adding an external project reference to a c# project. you just need to select ‘Database projects in the current solution’ and choose your database from the dropdown list.

You can also reference a system database here. Occasionally however you will need to reference a database that is neither of the above.  If the database you need to reference is already a data tier application you can browse for the dacpac but if it is not (for instance it may be a legacy database) you will need to create a data tier application from it first.

To extract a dacpac from an existing database, open the sql server object explorer in Visual Studio, connect to the server, right click the database and choose ‘Extract Data-tier application’.

Choose where to save the dacpac, then select ‘Extract Schema Only’ and decide whether you need the additional options such as user login mappings etc. Once you have created the dacpac you can go back to the add database reference dialog and select the dacpac you just created.

Database Location

In the Database Location section of the Add Database Reference dialog set the location of the referenced database e.g. ‘Different database, same server’, the name of the database (should be pre-filled) and whether you want to add a database variable to embed into your sql instead of a hardcoded path.  Finally (and often very importantly!) you may want to check the ‘Supress errors caused by unresolved references in the referenced project’ option at the bottom of the dialog.  This is especially useful in legacy databases that themselves reference other databases.

After clicking ok you should be good to go. This reference is there just to allow the application to compile, it does not get included in the deployment, but is an excellent way of making sure that your external references are correct.

As a final note, you need to ensure that your references are kept up to date if you are working on a project.  Although you can create a NuGet feed for your created dacpacs, as of Sept 2014 NuGet does not yet support importing to an SSDT project. For now you will have to keep a manual check.

How many times have you found inconsistencies between a test and live database, not known which one is right and been unsure who to ask on your team?  In my experience, especially in smaller teams with no DBA, this happens a lot and people do bad things like edit the live database directly (“Hmmm, according to the query plan I need to add an index.  I’ll just do that now.”).

By creating a SQL Server project using SSDT

• You will know exactly who changed what and when.
• You can fail a publish if database drift is detected (ie the database you are publishing to is different from the last time you published because someone changed something directly).
• You can create unit tests with standardised data that run against a fresh database.
• You can track database refactoring (ie changing a column name) so that tables are not dropped and rebuilt when published.
• You can fail a publish if data loss will occur.
• You can fully automate deployment to multiple servers and bring them all to a standard level regardless of their current version.

So much good stuff!

Here’s how you do it.

First, make sure you have the latest version of SSDT.

Create the project

1. In Visual Studio, select File / New Project.
2. Under Templates. click ‘SQL Server’ and choose ‘SQL Server Database Project’
3. Name your project according to the database you are importing and also name your solution, then click OK.
   

Import your database

1. Right click the project you just created and select Import, then Database
2. Select your database from the Source Database Connection dropdown list, or use the New Connection button to add it.
3. In addition to the default Import Settings that are selected I usually also select ‘Import Database Settings’ so that the correct database version is chosen in the project properties. Otherwise you can usually leave everything else as is.

Check your folders and files

At this point you will have a folder structure based on the database schema – if you only use dbo then you will only see that, otherwise you will see a folder per schema and an additional security folder where sql for creating your schemas has been put. (This is also where you put sql for new schemas) Here you can see that my database has two schemas – dbo and IntegrationTest. SSDT also separates out your functions, stored procs and tables. In the Security folder you can see the sql for creating the IntegrationTest schema.

If there is anything missing at this point you may want to check the import settings. You can always delete all the added files and folders and do the import again.

Build and be happy!

Right click your project and select Build.

• If there are no external references the project will build and create a dacpac. This will be saved in your bin directory in the appropriate configuration folder.
• If there are external references required you will need to add them. I have blogged how to do this here.

You can also add static data such as lookup tables or date dimensions and again I have blogged about this here.

We are now at a stage where you can version control your database and to publish it to a particular server you just need to right click the project and select ‘Publish’. Note that there are several options when publishing and you can create profiles for different servers.  I have blogged about how to do that here.

Good luck!

Here is the outline of how it all hangs together, I will do a few more detailed posts to follow up.

The solution itself consists of three projects. Two database projects and a unit test project.  They are checked into GIT (using BitBucket) and TeamCity watches the repository to trigger a build.  The build creates a dacpac for each database, publishes the changes to the appropriate server, runs the unit tests and saves the dacpacs into a nuget package that is published by TeamCity itself.

Out of the box, TeamCity can handle the build but you need MSTest and SSDT installed on the build server.  That is where your troubles begin.  You can download an MSTest agent as I wrote about here.  You can even install the SSDT tools for VS2012 (though that was a bit hit and miss for me). The real problem is that with VS2013 the SSDT tools are integrated and no longer available as a separate download.  In the end I had to settle for installing VS2013 on the build server.  Microsoft allows your developer license to be used on the build server and at least I can easily update the SSDT version – I’m not particularly happy about having to do it though.

In order to publish your database to the correct server you just need to include appropriate publish profile files and Microsoft have provided the ability in database unit tests to switch config files according to the build server machine name. The publishing part worked fine for me but the config switching seems flaky and if you put a whitespace in the wrong place your config will be ignored.  My workaround here was to resort to Slow Cheetah – a plugin from NuGet that allows config switching for app.config in the same way that you can achieve it with web.config. This works well but remember to restore your nuget packages as the first build step in TeamCity. It’s not so restrictive as a config that switches based on machine name (we only have a single build server so would be unable to test on multiple platforms).

The result is that on every check in, the dacpacs are built, deployed to a test environment, tested and if they pass, made available via a nuget feed for other projects.

It feels good when it all just starts working!

Go to SQLBits!

I can’t say it enough.  It is the best conference in Europe for learning SQL, it is extremely cheap. it is non-profit and it’s a lot of fun.  I’ve been going for the past 5 years and have never regretted it. This year was no exception, the conference itself runs from Thursday to Saturday with (this year) a pub quiz night (attended by several hundred people) on the Thursday and a big party on the Friday.  All of Saturday is free to attend and of course lunch and drinks are provided.

As a professional developer I consider it essential to attend conferences and try to make it to two or three a year, if your company considers itself professional too, it should pay for you to go. Speak to your manager, colleagues, friends, pets – whomever it takes – and get yourself a place at SQLBits next year, you won’t regret it.

Highlights from this year for me included Brent Ozars excellent session ‘Watch Brent Tune Queries’, David Morrisons ‘SQL Tips, Tricks and Misconceptions’ and Peter Ter Braakes ‘Power BI from Zero to Expert’.

See you there next year!