Showing posts with label SQL Server. Show all posts
Showing posts with label SQL Server. Show all posts

Thursday, November 4, 2010

TSQL Delimited List From Rows

Using TSQL, at some point you may need to create a column that contains a list of values that are normally rows in SQL Server. This blog entry will show you how to do that using the Chinook database (available at http://chinookdatabase.codeplex.com).

For sake of argument let's return a list of artists from the Artist table. First, a simple select query will return the following results...



The following TSQL code will return our delimited results.

Declare @ArtistAlbum varchar(1000)
Select @ArtistAlbum = coalesce(@ArtistAlbum + ', ', '') + Artist.Name From Artist
Select @ArtistAlbum

Wednesday, September 29, 2010

Introduction to Formatting Query Results as XML in Microsoft SQL Server 2008

We can use the FOR XML clause with SELECT statements in Microsoft SQL Server to produce XML output from our queries.  We might use this feature when creating output that will be consumed by an application that understands XML.  This post will demonstrate using this feature to create XML output of a contact list.


Here is the T-SQL used to create and populate our Contacts table:

CREATE TABLE Contacts
(
ContactId INT NOT NULL IDENTITY(1,1) PRIMARY KEY,
FirstName VARCHAR(30) NOT NULL,
LastName VARCHAR(30) NOT NULL,
PhoneNumber VARCHAR(20) NOT NULL,
EmailAddress VARCHAR(50) NOT NULL
);

INSERT INTO Contacts ( FirstName, LastName, PhoneNumber, EmailAddress )
VALUES ( 'Smith', 'Jon', '254-555-1234', 'jon@example.com' );


INSERT INTO Contacts ( FirstName, LastName, PhoneNumber, EmailAddress )
VALUES ( 'Rogers', 'Amanda', '254-555-4321', 'amanda@example.com' );


INSERT INTO Contacts ( FirstName, LastName, PhoneNumber, EmailAddress )
VALUES ( 'Gomez', 'Paul', '254-555-5252', 'paul@example.com' );


INSERT INTO Contacts ( FirstName, LastName, PhoneNumber, EmailAddress )
VALUES ( 'Peterson', 'James', '254-555-9876 ext 234', 'james@example.com' );

Let us assume that we have a contact management application that we wish to export this data to and that this fictitious application can import XML that is formatted like the sample below.


<?xml version="1.0"?>
<Contact-List>
<Contact>
<First-Name>John</First-Name>
<Last-Name>Doe</Last-Name>
<Telephone>800-555-1111</Telephone>
<E-mail>doe.john@example.com</E-mail>
</Contact>
<Contact>
<First-Name>Tommy</First-Name>
<Last-Name>Atkins</Last-Name>
<Telephone>800-555-2222</Telephone>
<E-mail>atkins.tommy@example.com</E-mail>
</Contact>
</Contact-List>


As you can see in addition to formatting the output as XML we will also need to map our table's columns to the XML format's elements: "FirstName" in the table will be "First-Name" in the XML.   We can use the FOR XML clause in our SELECT statement to shape the query's result to match the required format.  Below is the T-SQL used to create the XML.


SELECT 
FirstName AS 'First-Name',
LastName AS 'Last-Name',
PhoneNumber AS Telephone,
EmailAddress AS 'E-mail'
FROM Contacts
ORDER BY LastName, FirstName
FOR XML PATH('Contact'), ROOT('Contact-List');

Here is the result of that query.

<Contact-List>
  <Contact>
    <First-Name>Rogers</First-Name>
    <Last-Name>Amanda</Last-Name>
    <Telephone>254-555-4321</Telephone>
    <E-mail>amanda@example.com</E-mail>
  </Contact>
  <Contact>
    <First-Name>Peterson</First-Name>
    <Last-Name>James</Last-Name>
    <Telephone>254-555-9876 ext 234</Telephone>
    <E-mail>james@example.com</E-mail>
  </Contact>
  <Contact>
    <First-Name>Smith</First-Name>
    <Last-Name>Jon</Last-Name>
    <Telephone>254-555-1234</Telephone>
    <E-mail>jon@example.com</E-mail>
  </Contact>
  <Contact>
    <First-Name>Gomez</First-Name>
    <Last-Name>Paul</Last-Name>
    <Telephone>254-555-5252</Telephone>
    <E-mail>paul@example.com</E-mail>
  </Contact>
</Contact-List>

The FOR XML clause instructs the server to transform the results to XML.  The PATH parameter specifies that each record should be wrapped in an element named "Contact".  The ROOT parameter specifies that the result set should be wrapped in an element names "Contact-List".  Note that since the elements contain dashes and dashes are not permitted in column names an alias is needed in our query to name the elements to match the XML elements.   Note that element names in query such as 'First-Name' can be escaped by single quotes, square brackets, or double quotes.  All of the following are equivlement: 'First-Name', "First-Name", [First-Name].

The FOR XML has additional parameters that can be used to control the format of the output.  This post is intended as a basic introduction.

References

"Basic Syntax of the FOR XML Clause" from SQL Server Books Online





Wednesday, September 8, 2010

Enforcing Uniqueness on Optional Fields in SQL Server Database Tables

Enforcing the uniqueness of data in a table is a common requirement.  For example an "Employees" table might contain an "EmployeeID" field that must uniquely identify each row.  This is usually handled by creating a primary key or unique constraint on the field.  This works well when the field in question is required.  In our example each employee is assigned an employee ID number.  Sometimes however this is not sufficient to cover all circumstances.  Let us assume that each employee may be assigned a company email address, but not all employees will be given an email address.  No two employees can share the same email address.  We want the "EmailAddress" field in the Employees table either to be empty or contain a value not already in the table.  We can enforce the uniqueness of the email address values either by using an indexed view or by creating an index.


Here is the T-SQL used to create the Employees table we will use for our examples.

CREATE TABLE Employees
(
EmployeeID CHAR(4) NOT NULL PRIMARY KEY,
LastName VARCHAR(20) NOT NULL,
FirstName VARCHAR(20) NOT NULL,
EmailAddress VARCHAR(30) NOT NULL DEFAULT ''  
);

Option 1: Using an Indexed View
We can create a view of Employee records with an email address then create a unique index on the EmailAddress field.  If a user tries to insert a duplicate email address the insert will fail.

Here is the T-SQL used to create the view and index.   Note that the view only contains records which have a non-empty email address.

CREATE VIEW EmployeeEmailAddresses WITH SCHEMABINDING
AS
SELECT EmailAddress
FROM dbo.Employees
WHERE EmailAddress != '';

GO

CREATE UNIQUE CLUSTERED INDEX Index_EmployeeEmailAddresses_EmailAddress
ON EmployeeEmailAddresses ( EmailAddress );

We can then insert some records into the Employees table using the T-SQL statements below.

INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E100', 'Smith', 'Jon', 'smith@example.com' );

INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E200', 'Pond', 'Amy', '' );

INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E300', 'Noble', 'Donna', 'noble@example.com' );

INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E400', 'Jones', 'Martha', '' );

-- this insert will fail
INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E500', 'Smith', 'Sarah-Jane', 'smith@example.com' );

The last insert will fail because a record already exists with the email address "smith@example.com".  This approach will work in both SQL Server 2005 and 2008.

Option 2: Create an Index
SQL Server 2008 introduced the WHERE clause into the syntax for the CREATE INDEX statement.  We can create a unique index on records which are not empty with the T-SQL statement below.

CREATE UNIQUE INDEX Index_Employees_EmailAddress
ON Employees ( EmailAddress )
WHERE ( EmailAddress != '' );

To test this index first drop the indexed view and empty the Employees table, then insert our test data again.  The last insert will fail.

-- this will drop the indexed view we created earlier if it exists
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.VIEWS WHERE TABLE_NAME = 'EmployeeEmailAddresses' )
BEGIN
DROP VIEW EmployeeEmailAddresses;
END;

--  empty the Employees table
DELETE 
FROM Employees;



INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E100''Smith', 'Jon''smith@example.com' );

INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E200''Pond', 'Amy''' );

INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E300''Noble''Donna''noble@example.com' );

INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E400''Jones''Martha''' );

-- this insert will fail
INSERT INTO Employees ( EmployeeID, LastName, FirstName, EmailAddress )
VALUES ( 'E500''Smith''Sarah-Jane''smith@example.com' );


This technique is new to SQL Server 2008 and is not backwards compatible.  
For more information on each approach consult the SQL Server documentation.

References
Create View from SQL Server Books Online:

Create Index from SQL Server Books Online:

Wednesday, September 1, 2010

Using SET FMTONLY OFF in the Query string of Report Server

By default FMTONLY is set to ON in Report Server. When using a SQL statement containing temp tables an error message commonly occurs. You may get an message like "There is an error in the query. Invalid object name '#myTempTable'".



To fix this I’ve used the command SET FMTONLY OFF at the top of my SQL statement. This allows the SQL statement to return rows with data. The SET FMTONLY ON will only return column information.





Use the following SQL statement, changing the FMTONLY from ON to OFF, in Query Analyzer to see how this works.

USE AdventureWorks;
GO
SET FMTONLY ON;
GO
SELECT *
FROM HumanResources.Employee;
GO
SET FMTONLY OFF;
GO

Wednesday, August 18, 2010

Using the bcp Utility to Export Data to a CSV File with Microsoft SQL Server 2008

In previous blog posts I have demonstrated how data can be imported into a SQL server database table from a CSV file.  In this post I will introduce the bcp (Bulk Copy Program) utility that can be used to export the results of a query to a CSV file.  The bcp utility is included in the SQL Server installation and can be invoked from the command line or included in a batch file.  If you needed to create a CSV file in a specified directory once per day you could create a batch file that includes a call to bcp and schedule the batch file to run daily.

Below is an example of bcp command export a list of employee names and phone numbers to a CSV file.   The command could be executed from the command line or as part of a batch file
.

bcp "SELECT LastName, FirstName, HomePhone FROM Northwind.dbo.Employees ORDER BY LastName, FirstName" queryout "employees.csv" -T -S . -w -t,

The first part "bcp" is the name of the executable program.  When using it at the command line or in a batch file make sure that its parent directory is included in your path variable.  Next in quotes is the SELECT statement that will generate the results. I am using the sample Northwind database.  I have included the database name "Northwind" and schema "dbo" in my FROM clause.  The queryout parameter instructs the bcp utility to export the query results to a file specified by the next parameter "employees.csv".  The –T specifies that integrated authentication will be used.  If we wanted to use SQL authentication this is also supported.  The  –S parameter specifies the server, in this case the local machine. The –w parameter instructs bcp to treat the data as Unicode characters.  The –t parameter specifies that a comma be used as the field delimited instead of the default tab character.  Note that command arguments are case-sensitive "-t" is not the same as "-T".


Here is a screen shot of the employees.csv text file.


You can see from the screenshot that the files created using the bcp utility do not include column headings. 

This post is a basic introduction to the bcp utility. The bcp utility has other capabilities: it can be used to import data, similar to the BULK IMPORT statement, and it can also be used to create format files.

References

Bcp Utility from SQL Server Books Online:

Wednesday, July 28, 2010

Using an XML Format File with Bulk Import in SQL Server 2008

In a previous blog post (http://wardlawclaims.blogspot.com/2010/07/introduction-to-sql-server-2008-data.html ) I provided an introduction to the BULK INSERT statement which can be used to import the contents of a CSV file into a table in a SQL Server database.  In this post I will demonstrate how a format file can be used when the columns in the source file are ordered differently than those in the target table.  Below is the T-SQL used to create the target "Contacts" table:

CREATE TABLE Contacts
(
  FirstName VARCHAR(30) NOT NULL,
  LastName VARCHAR(30) NOT NULL,
  PhoneNumber VARCHAR(25) NOT NULL
);

Next is a screen shot of the source CSV file.


While the Contacts table has columns named "FirstName" and "LastName", the CSV file has corresponding columns named "Given name" and "Surname".  The CSV file also contains an additional column, "Date of Birth" which needs to be ignored since no corresponding column exists in the Contacts table.  The default behavior of the BULK INSERT command expects that the columns in the source and target appear in the same order.  To avoid this requirement I will use a format file to specify how the columns in the source file will map to the columns in the target table.

Below is the format file's contents, note that I am using an XML format file.


<xml version="1.0" >
<BCPFORMAT 
  xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <RECORD>
  <FIELD ID="LastName" xsi:type="CharTerm" TERMINATOR="," />
  <FIELD ID="FirstName" xsi:type="CharTerm" TERMINATOR="," />
  <FIELD ID="BirthDate" xsi:type="CharTerm" TERMINATOR="," />
  <FIELD ID="PhoneNumber" xsi:type="CharTerm" TERMINATOR="\r\n" />
  </RECORD>
  <ROW>
  <COLUMN SOURCE="FirstName" NAME="1" xsi:type="SQLVARYCHAR" />
  <COLUMN SOURCE="LastName" NAME="2" xsi:type="SQLVARYCHAR" />
  <COLUMN SOURCE="PhoneNumber" NAME="3" xsi:type="SQLVARYCHAR" />
  </ROW>
</BCPFORMAT>

The "RECORD" element corresponds to the source file.  Each "FIELD" element maps to a column in the CSV file.  The "ROW" element corresponds to the target table.  Each "COLUMN" element corresponds to a column in the target table.  Both FIELD and COLUMN elements are listed in the format file in the same order they appear in the source file and target table.


Each FIELD has two required attributes: "ID" and "type".  The ID attribute uniquely identifies each FIELD element and the type specifies the data type contained in the column.  The ID value is arbitrary; it does not need to match the column name specified in the source file.  In this case I have substituted "LastName" in the format file for "Surname" in the source file and so on.  I have also included the "TERMINATOR" attribute to specify that columns are delimited by commas, except for the final column in each row which is terminated by a Windows new line.


Each COLUMN has two required attributes: "SOURCE" and "NAME".  The SOURCE value matches the ID value of the FIELD that the SOURCE row corresponds to.  In this case the FirstName column appears first in the target table so it is listed first in the ROW element.  The NAME value is an arbitrary value to uniquely identify each COLUMN element.  The "type" attribute is optional and specifies the data type for the column in the target table.  All columns in the target table are of type VARCHAR so the format file type of SQLVARYCHAR is used.  Because the target table does not contain a column to store the date of birth there is no COLUMN that corresponds to the BirthDate FIELD.


Here is the T-SQL statement used to execute a bulk import using an XML format file.

BULK INSERT Contacts 
FROM 'C:\sampledata\source.csv' 
WITH  
(
  FORMATFILE = 'C:\sampledata\contacts_format_file.xml',
  FIRSTROW = 2
);

The FORMATFILE value is the path to the XML format file and the FIRSTROW value of 2 instructs the server to ignore the first record in the source file which contains our column headings.


Below is a screen shot of the contents of the "Contacts" table after the import.


The format file I used in this post was in XML format, you can also use the bcp (Bulk Copy Program) utility included with SQL Server to create a plain text format file.

References

Schema Syntax for XML Format Files from SQL Server Books Online:

The BULK INSERT statement from SQL Server 2008 Books Online: 

The sample data was created using Benjamin Keen's online data generator:

Download the sample CSV file:

Download the XML format file:

Wednesday, July 7, 2010

Introduction to SQL Server 2008 Data Import with BULK IMPORT

There are several methods for importing data from a text file, such as a comma or tab separated values file, into a SQL Server database. This post will provide a brief walk though demonstrating the basics of the BULK INSERT statement. The BULK INSERT statement can be used in scenarios where you need to import the contents of a text file into a table. For example, you might receive new records in batches rather than individually. In this scenario it makes sense to import the batch at once rather than executing an insert for each record in the batch. In this post I will import a list of contacts containing names (first and last) and phone numbers into a table named "Contacts". I will make text file used in the post available for download.

Here is the T-SQL used to create the Contacts table.


CREATE TABLE Contacts
(
FirstName VARCHAR(30) NOT NULL,
LastName VARCHAR(30) NOT NULL,
PhoneNumber VARCHAR(25) NOT NULL
);

I have a CSV (comma seperated values) text file containing two handle sample contact entries that I wish to import. The first row contains the column headings and each subsequent row contains the three column values separated by commas. Below is a sample of the file's contents.



In order to import these records into my Contacts table I will use SQL's BULK INSERT statement as it appears below:

BULK INSERT Contacts
FROM 'C:\sampledata\source.csv'
WITH
(
FIELDTERMINATOR = ',',
FIRSTROW = 2
);
The destination table "Contacts" is specified first after the "BULK INSERT" keywords. The path to the CSV target file is specified next after the "FROM" keyword. The "WITH" clause allows us to customize how the data will be imported. In this case I have used the "FIELDTERMINATOR" option to indicated that a comma is the column separator for our file. The default is the tab character. I have used the "FIRSTROW" option to skip the first line of the CSV file since it contains column headings which do not need to be imported.

Below is a screen shot of the contents of the "Contacts" table after the import.


This is a very basic example. In this case the Contacts table and the CSV file contain exactly the same columns in exactly the same order. If there were differences between the layout of the table or CSV file additional options would need to be specified in the "WITH" clause.

References

The BULK INSERT statement from SQL Server 2008 Books Online:


The sample data was created using Benjamin Keen's online data generator:
http://www.generatedata.com