Automating Citrix Provisioning Server Install with RES AM

Here is a blog post I put together on automating the build of Citrix Provisioning Services using RES Automation Manager 2012. Before we get into the details I thought I’d mention a few resources and solutions I found on the way which helped me out. A big thanks to:

Before you can begin you will need to make sure you have the following prerequisites in place:

  • Provisioning Server Software (PVS 6.1 used for this example);
  • Windows Server 2003 upwards (Windows 2008 R2 SP1 used in this example);
  • NET 3.5 or higher is installed;
  • RES Automation Manager 2012;
  • Use the latest Citrix Licensing server.

I’ve split the automated process in to two distinct parts; creating the PVS database and installing PVS to make it easier to digest. If you’re lazy or just want to crack on you can just download the building blocks and get going! Note: you will need to update the resource references to the PVS 6.1 installation files.

Creating the PVS Database

Before you can automate the PVS installation we need to have a database in place for the PVS servers to connect to. Unfortunately for us there’s not an easy way to accomplish this as we need to generate an SQL script with our required database values. As we’re invoking the creation process from RES Automation Manager 2012 we can utilise parameters so we can prompt the administrator for these values at run time.

To create the SQL script we first need to install the Provisioning Services software on a clean Windows 2008 R2 server or if you have an install already you can obtain from here. Once installed we can run C:\Program Files\Citrix\Provisioning Services\DBscript.exe to launch the Provisioning Services Database Script Generator. Exciting stuff I know !!!


If we complete the details with placeholders (as above) for the database name and farm name, DBscript will create the required .SQL script with values that we can use within our RES Automation Manager jobs. Click OK and it will create the CreateProvisioningServerDatabase.sql file in the path specified, complete with embedded placeholders.

We can now import this file as a resource into the RES Automation Manager console. Note: remember to tick the ‘Parse Environment variable and parameters’ checkbox. If you forget to do this we’ll attempt to create a database with a name of $[PVSDB] which probably won’t work (not that I’ve checked!).

To create the required SQL database we can utilise the CreateProvisioningServerDatabase.sql file with the built in RES Automation Manager database connector task(s) or via SQLCMD on the local Microsoft SQL instance. As we’re cheap and can’t assume that you’re licensed for the relevant connector, we’ve utilised SQLCMD in the building blocks. For more details on this, download them and have a look.

After the database has been created we need add SQL permissions to the database (if using a network user for the SOAP and STREAM services). This is achieved with a couple of SQL statements (see the building blocks for more information). If we’re using an Windows service account to run these services, the user will be configured later during the install… And now the fun begins;

Installing and Configuring PVS

Now that the database is created we can move on to installing the software, configuring and adding servers to the farm. Installing the software is no problem however configuring and adding servers to the farm is a bit more involved. The method I used for configuring the servers was by utilising the configwizard.ans file which holds all the configuration items. By running the %PROGRAMFILES%\Citrix\Provisioning Services\configwizard.exe /s the answer file is in turn created here C:\ProgramData\Citrix\Provisioning Services\configwizard.ans.

Once we have the configwizard.ans file we can edit it and embed our RES Automation Manager 2012 parameters within it. If you’d like to know what options can be configured in the answer file, run configwizard.exe /c. The configuration wizard will write a C:\ProgramData\Citrix\Provisioning Services\configwizard.out file. Again, all this information is in our building blocks.

I used two different answer files one for the first server joining the farm and the other for all subsequent servers. Below is an example of the first server configwizard.ans file:

DatabaseInstance= FarmExisting=$[PVSFARM]
BootstrapFile=C:\ProgramData\Citrix\Provisioning Services\Tftpboot\ARDBP32.BIN

Once the answer file/files have been created and modified, import them into the RES Automation Manager resources. Note: remember to select the ‘Parse Environment variable and parameters’ checkbox!

Finally to automate the actual PVS install, we need to make sure we download these resources to the C:\ProgramData\Citrix\Provisioning Services\ directory on the target server. Then we kick off the configuration wizard which will apply the configuration, by running configwizard.exe /a. Once complete the services should start automatically and when you start the PVS console and connect you should be presented with the new farm, well hopefully anyway !!

Problems Encountered

If you do have problems using the answer file and the install fails the best place to start troubleshooting is under C:\ProgramData\Citrix\Provisioning Services\Log directory. If all goes wrong you will notice that there will be only one file here;  configwizard.log. And at the end of this file hopefully it should give you some meaningful reason as to the failure. If all works fine and the services start you should see around 8 Log files and have a big smile on your face :D.

I did have other issues whilst getting this to work. Here are a few notes in case they help:

  • No device License available when a new machine is booted using provisioning server you will see the error in the streamprocess log on the PVS server and also on the device a pop message will say “No device License currently available for this computer a system shutdown will be initiated in 96 hours. I found the resolution to this problem was to upgrade the license server to the latest build.
  • PVS Console install does not install via AM job – ensure that UAC is disabled and use a security context to run the job instead of the local System account.
  • After a server install I could not mount Vdisks on PVS server and might get an error similar to “Cannot mount Vdisk mapi error”. Looked at device manager and noticed that the Citrix virtual hard disk Enumerator driver was not installed correctly. To resolve this first remove the device and then go to %PROGRAMFILES%\Citrix\Provision Services\Drivers right hand click and install cfsdep2.inf and then go back to device manager and add legacy hardware and select “I have disk” and then point to same location and the file is cvhdbusp6.inf. It should then hopefully install this device without any issues. Or the Preferred option with RES AM create a module to download the following, CFSDep2.inf and CFSDep2.sys to C:\windows\system32\drivers before installing provisioning server and all should be okay.
  • When using a service account make sure that this user is given the required permissions i.e read/write on the PVS store directory on the PVS servers / db_datareader and db_datawriter on the database although the latter can be done if you select configure user for database.

Building blocks now updated as there was a problem with the Service Account password passing through to the answer file, this should be resolved. Have also added a module to remove the answer file as the password is in plain text.

Hope this helps, Enjoy ! Smile Simon

PVS Image Management

There has been a bit of banter on the Twittersphere about how people manage and document their PVS images. It was suggested (by more than just me Smile) that RES Automation Manager could be utilised for this task. This post is not a best practice guide as to how to create, update or document your images, rather a use case on how and why we use/recommend RES Automation Manager. Heaven forbid, you might even decide to do away with Provisioning Services as a result. Either way RES Automation Manager will play very nicely with or without PVS but I couldn’t fit it into 140 characters!

Provisioning Services Private Image Management

Maintaining the gold or private mode PVS image can be a complex task for a number of reasons. Simplifying any of these potential hurdles can only be a good thing, right?

  1. A certain level of skill is required to both create and maintain images. There are numerous tasks that need to be completed and in some cases, performed in a particular order. As a result, this task is typically left or assigned to the senior administrators.
  2. Application upgrades can taint or stain the master image. Some applications require an uninstallation of the old version and installation of the new product MSI. I don’t need to tell you that uninstallers are not always reliable or clean everything out when run!
  3. How are changes to the gold image documented to ensure that they’re incorporated into all other PVS images? It is typical that there will be more than one image for deployment. For example, hardware differences will typically require separate images.
  4. Ad-hoc and emergency changes can wreak havoc with your PVS images. How quick and easy is it to push an update out to 100 XenApp servers streamed from a central image? If we make changes whilst the servers are running then they’ll be lost when the write cache is erased meaning we either have to reapply this change after every reboot or update our gold image pronto! This will get a lot more interesting if the servers are rebooted on a nightly basis and the write cache cleaned!

RES Automation Manager

If know me by now, you probably know that I’m going to say that RES Automation Manager is the answer to all your prayers! Now whilst it can certainly address the above “issues” (and I would recommend it in conjunction with Provisioning Services any day of the week) there are other processes and solutions that may address one or more of the above and deploying RES Automation Manager won’t automagically fix them. A good example of this is documentation. If your internal processes mandate that all changes are documented and you bypass this process, there is nothing to stop you bypassing this process even if Automation Manager is installed!

What RES Automation Manager Won’t Do

Thought I’d better get this bit out of the way before you get all the way to the end and are disappointed! RES Automation Manager is a Run Book Automation tool and not an imaging/deployment tool. This means that we cannot (directly) deploy an Operating System from RES Automation Manager. Fortunately for us there are many technologies out there that can, e.g. Windows Deployment Services/Microsoft Deployment Toolkit which BTW can by combined with RES Automation Manager – take a look at this White Paper. Why reinvent the wheel?!

What RES Automation Manager Will Do

So once we have our Operating System deployed and the RES Automation Manager agent installed (we can do this with WDS/MDT as mentioned earlier) what benefits will this give us? Well, at a simplistic level, RES Automation Manager can automate the entire server configuration and application deployment process. This process can also include installing XenApp and XenApp Prep as well as any other applications. This obviously takes some additional time but gives us a clean, repeatable process for deploying a XenApp server from scratch. It’s a strategic decision and not a tactical one!

Why is this important? Typically it comes down to issues #2 and #3 so let’s take them one at a time..

Issue #1

RES Automation Manager can reduce this complexity by removing Provisioning Services altogether. I’m not suggesting that you remove this from your infrastructure. Not even for one minute. However, if you don’t need to have a clean image after every reboot getting shot of PVS maybe an option? We have automated the complete server deployment and can typically provision a new server in a few hours from start to finish; Operating System, XenApp and applications. OK it’s a few hours of time, but there is no user interaction required. I’m guessing that it’s probably not that often that you need to add a new server within 30 minutes?

This benefits the typical IT department as these are now regular servers. They’re supported in the same way as other servers and they have a proper OS install etc. There are downsides too. Now we need to patch and maintain multiple OS instances and not just one master image. Isn’t this part of the reason you deployed Provisioning Services in the first place?

Issue #2

By having a repeatable process for building our XenApp server(s) from scratch we can avoid tainting our image. If we need to cut a new image then we can deploy a completely clean server and deploy the required applications as required. We don’t need to uninstall and reinstall or upgrade applications. I’m not advocating this as a best practice, but I know lots of admins that are a lot happier with this process. It doesn’t need to be performed for all updates, but you now have an option as to whether you update the master image or cut a new one. If you have not automated the entire deployment and configuration process, recreating a new image from scratch probably doesn’t make you feel warm and fluffy inside!

When you finally get run over by a bus (it’s going to happen one day as everyone keeps saying it) pretty much anyone with ounce of intelligence can deploy a new server or reverse engineer the Modules and Tasks in the Run Book to discover how things are tied together.

Issue #3

By virtue of automating the entire configuration and deployment process with RES Automation Manager, you have actually documented every step in the process. RES Automation Manager includes the ability to create an Instant Report of any or all Run Books, Projects and/or Modules. These reports are very detailed (small example here) and typically run to 1,000+ pages. For us consultants, this feature alone is worth its weight in gold. Did I mention that it’s available in RES Workspace Manager too? Winking smile

Issue #4

Finding out when and by whom the changes were made. Whether they be changes made to the gold image or Ad-Hoc emergency it doesn’t matter the audit trail of the changes is vitally important especially with change management processes. Well would be surprised to hear that RES Automation Manager has an in-built Audit Trail which allows you to view all actions performed in RES Automation Manager – how handy is that when a witch hunt is on (Oh that never happens now does it!?).

Issue #5

As usual I’ve saved the best until last and you didn’t see number 5 coming! The “pièce de résistance” if you like. This might get a bit confusing so strap yourselves in ready…

Emergency changes to running PVS instances are pain. Depending on your configuration after a reboot changes may be lost and depending on your requirements, you may reboot nightly or even weekly. If there is a configuration change that needs to be made then ultimately we need to update the master image. We can implement the change on the running instances, but it will be lost at some point when the write cache is cleared. Until the master image is updated we will need to implement the change, potentially after every reboot.

Because RES Automation Manager is a Run Book Automation tool we can implement this change across all running instances within minutes. “WAIT!”, I hear you cry, “These changes will be lost after a reboot!” Correct. But now we have achieved two things; documented the change and can automate the update to the master image at some point in the future.

Why did I say at “some point in the future?” Fortunately for us there is a hidden gem within RES Automation Manager called Snapshot Intelligence. With a name like that it better be good right!?

As the RES Automation Manager database has a record of all jobs that have executed on a given agent it can detect a snapshot. Whether this is a virtual snapshot or a backup restoration, it makes no odds. In our PVS world, if RES Automation Manager jobs have been run on a machine and the PVS instance is reset back to our master image state (write cache cleared), RES Automation Manager will detect this as a snapshot. You with me so far..?!

Once a snapshot is detected, RES Automation Manager can automatically reapply the job history (I’ll pause whilst you take this in and wait for the penny to drop!).

So if we automate all the emergency or ad-hoc updates with Automation Manager we can automatically reapply these after every reboot? Yes. No need to update the master image for every change? Yes.

In fact it gets better than that. When we update our master image we can run the exact same job history (automatically if you wish) to update the gold image. If you want to cut a new image from scratch we’ve got that covered too. Above all, if everything is automated with RES Automation Manager it’s automatically documented too. Needless to say, you get all the usual audit logging and change history.


So, in summary, using RES Automation Manager in combination with Citrix Provisioning Services has huge benefits, but there’s obviously a cost associated. Would I recommend it? Absolutely! For all of the above reasons. Is it worth it? Unfortunately, I can’t tell you that as only you know your environment.

Can RES Automation Manager replace Provisioning Services? Not entirely as you’ll still need WDS/MDT (or equivalent) to deploy the OS. It also depends on your reasons for deploying PVS in the first place. If it’s for near instant deployment, remove local disks, reduce the storage footprint or a clean image on every reboot, you’ll probably be using it for a long while yet. If your reasons are purely for “single image” management then you could potentially replace PVS in favour of a “traditional” deployment. Would I recommend this? It depends!

I know we’ve been focused on Provisioning Services in this article but RES Automation Manager will help you with the rest of your infrastructure automation. Desktops, laptops, servers; Exchange and Active Directory etc. You may have XenDesktop, Quest vWorkspace or VMware View for your virtual desktops. The same principal applies and you may even be using PVS in combination with these. Anyway, I don’t need to preach to the converted!

I will say that it should be a strategic decision to deploy RES Automation Manager. Don’t underestimate the amount of time it takes to automate and test. But I guess you already spend a lot of time testing your images?

You can find some video overviews/introductions on RES Automation Manager on Citrix TV and RES Tutorials. If you don’t want to take the time to download, install and configure RES Automation Manager but want to take a quick look, you can always request access to the RES Showcase. Some background and example videos on the Showcase platform can be also found here.

I’ll get off my soapbox now and crawl back to whence I came! Please feel free to comment and I’d love to hear your thoughts. Iain