How to process a cube in a SQL Agent job

It can be done but it's not well documented and it's complicated by the fact that you can't easily get the result of the process. Unless you actively check and manually raise an error if there's been a problem, the job will report sucess no matter how screwed up the cube processing is. That part is all about the XML results that are output by the process.

Here's how I do it (and I'd be very interested if you have any suggestions to improve it):

Connect to the Analysis Services server in SQL Server Management Studio.

Right click on the database and select Process.

Configure all the options and then use the Script button to grab the XML that defines the processing job. It will look something like this:

<Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">

    <ErrorConfiguration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2">

        <KeyErrorLimit>-1</KeyErrorLimit>

        <KeyErrorLogFile>\\rep01\joblogs\ProcessASOnBI1KeyErrors.log</KeyErrorLogFile>

        <KeyNotFound>ReportAndContinue</KeyNotFound>

        <KeyErrorAction>ConvertToUnknown</KeyErrorAction>

        <KeyErrorLimitAction>StopProcessing</KeyErrorLimitAction>

        <KeyDuplicate>IgnoreError</KeyDuplicate>

        <NullKeyConvertedToUnknown>IgnoreError</NullKeyConvertedToUnknown>

        <NullKeyNotAllowed>ReportAndContinue</NullKeyNotAllowed>

    </ErrorConfiguration>

    <Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2">

        <Object>

            <DatabaseID>BI1</DatabaseID>

        </Object>

        <Type>ProcessFull</Type>

        <WriteBackTableCreation>UseExisting</WriteBackTableCreation>

    </Process>

</Batch>

Create a new SQL Agent job.

Add a step to get rid of any old output file(s). Change the step type to CmdExec and paste in this code, modifying the file name:

if exist "\\rep01\joblogs\ProcessASOnBI1Log.xml" (del "\\rep01\joblogs\ProcessASOnBI1Log.xml")

Add a step for the processing. Change the step type to SQL Server Analysis Services Command and paste in the XML. Go to the Advanced page and give it an XML output file - with the same name you used in the previous step - to write its results to.

Add a step to check the output XML file to see whether the process was successful. Change the step type to T-SQL and paste in this code:

-- Check the result files from the cubes

 

set nocount on;

 

create table #File

(

        ResultFileName        varchar(255)

);

insert #File (ResultFileName) values ('\\rep01\joblogs\ProcessASOnBI1Log.xml');

insert #File (ResultFileName) values ('\\rep01\joblogs\ProcessASOnBI2Log.xml');

 

create table #FileContents

(

        LineNumber        int identity

        , LineContents        nvarchar(4000)

);

 

declare @FileContents        nvarchar(4000)

        , @NewLine        char(2)

        , @FileName        varchar(255)

        , @CubesFailed        bit

        , @CmdLine        varchar(300)

 

set @NewLine = char(13) + char(10);

 

set @CubesFailed = 0;

 

-- Loop through result files

declare file_cursor cursor

local forward_only

for

select        ResultFileName

from        #File;

 

open file_cursor

 

        fetch next from file_cursor

        into         @FileName

 

 

        while @@fetch_status = 0

        begin

 

                set @CmdLine = 'type ' + @FileName;

 

                insert #FileContents

                exec master.dbo.xp_cmdshell @CmdLine;

 

                select        @FileContents = isnull(@FileContents, '') + @NewLine + isnull(LineContents, '')

                from        #FileContents;

 

                select @FileName;

 

                select @FileContents;

 

                set nocount off;

 

                if @FileContents like '%error%'

                begin

                        set @CubesFailed = 1

                end

 

                delete #FileContents;

 

 

                -- Get the next cursor row

                fetch next from file_cursor

                into         @FileName

 

        end

 

close file_cursor

deallocate file_cursor

 

 

drop table #FileContents;

drop table #File;

 

 

if @CubesFailed = 1

begin

        raiserror('Cubes failed to build.', 16, 1)

end

Modify it to add your own file name(s) into #File at the top. This step will raise an error if the processing failed. If you don't do this, you won't know whether the job has worked.

 

Comments

# danh said:

Ah - golf and BI,  my twin loves !  = )

With XMLA I have had problems in the past with degenerate fact dimensions not processing before the cube, leading to Key violations.

Which led to having to script out the individual dimensions, which is by ID, which I have sometimes forgotten to change, leading to non-intuitive id's in the XMLA...

Because of this I now create an Integration Services job which has a few steps to process the dimensions and then the cubes.  Save as file, add to a Job as an Integration Services step

Also allows you to use configurations between platforms...however the error handling is a bit binary.

What do you think ?

26 November 2009 08:00
# DavidWimbush said:

Hi Dan,

Golf - good spot! I thought I'd sanitised it. I haven't that kind of dimension trouble so far but now I'll know how to handle that problem if I get it. Thanks.

I haven't really looked at SSIS at all. I had a bad experience with DTS when I was young, then they ripped that up and replaced it (so I was glad I hadn't been relying on it), and now it's all visual designers (and I'm more of a script man). Perhaps I ought to give it a chance but I worry about being able to assess the impact of schema changes etc.

How do you find SSIS? Is it really worth the time and effort to learn it?

26 November 2009 10:24
# danh said:

Hi David

Well, with SSIS you have to go in with a couple of thoughts ;

It certainly isn't as easy as DTS.

It's still very easy for people to write bad packages, list many MS database focussed produces best practice is opaque.

DTS is very reliable IMO if you stick within the confines of doing what you should be doing and don't start sticking VBScript and dynamic operations all over the place.

Like all ETL problems, a good dose of T-SQL helps a lot.

So with SSIS, stick to the following first steps which I was given by a luminary and have been invaluable ;

1 Use expressions based on a string variable for datasources.

(Have to set a property on them that they are not prevalidated for this to work.)

2 Use a DTSConfig configuration file to get the datasource string for your local database to get other settings.

3 Keep other configs in a configuration table, eg your AS database server and name.

4 In your package setup Analysis Services Processing tasks to do dimensions and then cubes in parallel streams - easier to work out what may happen in parallel and which won't than XMLA, and non-id sensitive.

5 Deploy your packages as files, not into msdb  metadata.  use source control on them (eg SVN or similar)

6 Point IS job steps at the package file.

7 Te Voila

Sounds hideous, but it isn't really and well worthwhile.

Also, I find DBA's are happier with something they can see as opposed to an opaque XMLA file.

Happy to elucidate further if you want.

dan

29 November 2009 11:10
# samikane said:

Here are some other articles on SSAS Processing: ssas-wiki.com/.../Articles

23 February 2011 16:32
# DavidWimbush said:

Thanks, samikane. I hadn't heard of that project. I'll be digging into that!

23 February 2011 16:51