This is part 3 of a short series on developing service applications in SharePoint 2010:
- Part 1: Getting Started with SAF + Starter Solution (with Logging / Configuration / Service Locator)
- Part 2: Custom Service Application – Logical Components
- Part 3 (This Post): Custom Service Application – Base Solution
- Part 4: Custom Service Application – Full Infrastructure (with Admin UI and PowerShell)
- Part 5 (Final): Custom Service Application – Integrating features and capabilities into your service application (sample MailChimp integration)
In this post, we are going to explore the guts of our custom service application (as usual, the source code is provided at the end of the post). As promised in my last post, there will be lots of code in this one, so hang on tight, and bare with me. As I mentioned in my first post in this series, the goal of this Service Application development exercise is to build an infrastructure within SharePoint 2010 for your company to add value to your SharePoint investment, by exposing 3rd party and custom capabilities and features to your farm(s) in a scalable and maintainable way. I recognize this isn’t for everyone, but if you’re part of a mid-size to big company, I think there is a lot of merit to consider this approach. This is NOT just for ISVs, but ISVs could also take advantage of this (if they aren’t already).
Let’s say you have X number of 3rd party systems you want to integrate into SharePoint. A company service application infrastructure can provide a standard, consistent, and scalable mechanism to embed each system into SharePoint and make it available throughout the farm. You can then also use this same infrastructure to build full custom applications as well if you’d like.
In this post, we are going to create the Service Application infrastructure, and in the next post we will embed a sample 3rd party system into the service application and expose it to the farm.
The following picture (the Visual Studio solution) illustrates the essence of the Service Application infrastructure that I coded up this week for this post (you can download this at the end of this post).
There is too much code to go over in this one post (it was actually a decent amount of work), so I’ll try to discuss the major classes and how it’s all put together, feel free to jump ahead and take the code for a spin yourself.
Primary Classes
MCService
The MCService class is the top level parent object for the Service Application. This class is used to create and update the service application and service application proxy.
using System; using System.Runtime.InteropServices; using System.Security.Permissions; using Microsoft.SharePoint.Administration; using Microsoft.SharePoint.Security; namespace MyCorp.SP.ServiceApplication { [Guid(Constants.ServiceGuid)] [SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true)] public class MCService : SPIisWebService, IServiceAdministration { #region Constructor public MCService() { } public MCService(SPFarm farm) : base(farm) { } public MCService(string name, SPFarm farm) : base(farm) { } #endregion #region Typical Property and Method Overrides public override string TypeName { get { return Constants.ServiceTypeName; } } public override string DisplayName { get { return Constants.ServiceDisplayName; } } public override SPAdministrationLink GetCreateApplicationLink(Type serviceApplicationType) { if (!ValidateApplicationType(serviceApplicationType)) return null; return new SPAdministrationLink(Constants.ServiceApplicationCreateLink); } #endregion #region IServiceAdministration Implementation [SharePointPermission(SecurityAction.Demand, ObjectModel = true)] public SPServiceApplication CreateApplication(string name, Type serviceApplicationType, SPServiceProvisioningContext provisioningContext) { ArgumentValidator.IsNotEmpty(name, "name"); ArgumentValidator.IsNotNull(serviceApplicationType, "serviceApplicationType"); ArgumentValidator.IsNotNull(provisioningContext, "provisioningContext"); ArgumentValidator.IsNotEqual(serviceApplicationType, typeof (MCServiceApplication)); var applicationPool = provisioningContext.IisWebServiceApplicationPool; // If an application with the same name already exists, return it (do not create a new one with the same name) var application = MCServiceUtility.GetApplicationByName(this, name); if (application != null) return application; var databaseParameters = SPDatabaseParameters.CreateParameters(Constants.ServiceApplicationDefaultDatabaseName, SPDatabaseParameterOptions.GenerateUniqueName); databaseParameters.Validate(SPDatabaseValidation.CreateNew); return MCServiceUtility.CreateServiceApplication(this, name, databaseParameters, applicationPool, Constants.ServiceApplicationDefaultEndpointName); } // This method is NOT part of the IServiceAdministration interface, but we are adding it in order to // provide more granular control over service application creation. We will use this method to create the service // application from our central admin UI. [SharePointPermission(SecurityAction.Demand, ObjectModel = true)] public SPServiceApplicationProxy CreateProxy(string name, SPServiceApplication serviceApplication, SPServiceProvisioningContext provisioningContext) { ArgumentValidator.IsNotEmpty(name, "name"); ArgumentValidator.IsNotNull(serviceApplication, "serviceApplication"); if (!ValidateApplicationType(serviceApplication.GetType())) throw new NotSupportedException(); return MCServiceUtility.CreateServiceApplicationProxy(this, (MCServiceApplication) serviceApplication, name); } public SPPersistedTypeDescription GetApplicationTypeDescription(Type serviceApplicationType) { if (!ValidateApplicationType(serviceApplicationType)) throw new NotSupportedException(); return new SPPersistedTypeDescription(Constants.ServiceApplicationTypeName, Constants.ServiceApplicationDescription); } public Type[] GetApplicationTypes() { return new[] {typeof (MCServiceApplication)}; } [SharePointPermission(SecurityAction.Demand, ObjectModel = true)] public MCServiceApplication CreateApplication(string name, string dbServer, string dbName, string dbUserName, string dbPassword, string failoverInstance, SPIisWebServiceApplicationPool applicationPool, string defaultEndpointName) { ArgumentValidator.IsNotEmpty(name, "name"); ArgumentValidator.IsNotEmpty(dbServer, "dbServer"); ArgumentValidator.IsNotEmpty(dbName, "dbName"); ArgumentValidator.IsNotNull(applicationPool, "applicationPool"); ArgumentValidator.IsNotEmpty(defaultEndpointName, "defaultEndpointName"); // If an application with the same name already exists, return it (do not create a new one with the same name) var application = MCServiceUtility.GetApplicationByName(this, name); if (application != null) return application; var databaseParameters = SPDatabaseParameters.CreateParameters( dbName, dbServer, dbUserName, dbPassword, failoverInstance, SPDatabaseParameterOptions.None); return MCServiceUtility.CreateServiceApplication(this, name, databaseParameters, applicationPool, defaultEndpointName); } // This method is NOT part of the IServiceAdministration interface, but will use the following method to allows // administrators to modify the service application from the central admin UI. [SharePointPermission(SecurityAction.Demand, ObjectModel = true)] public MCServiceApplication UpdateApplication(MCServiceApplication serviceApplication, string newName, string dbServer, string dbName, string dbUserName, string dbPassword, string failOverServerInstance, SPIisWebServiceApplicationPool applicationPool, string defaultEndpointName) { ArgumentValidator.IsNotNull(serviceApplication, "serviceApplication"); ArgumentValidator.IsNotEmpty(serviceApplication.Id, "serviceApplication.Id"); ArgumentValidator.IsNotEmpty(newName, "newName"); ArgumentValidator.IsNotEmpty(dbServer, "dbServer"); ArgumentValidator.IsNotEmpty(dbName, "dbName"); ArgumentValidator.IsNotNull(applicationPool, "applicationPool"); ArgumentValidator.IsNotEmpty(defaultEndpointName, "defaultEndpointName"); // Make sure the service application exists serviceApplication = MCServiceUtility.GetApplicationById(this, serviceApplication.Id); if (null == serviceApplication) throw new NullReferenceException("Service application does not exist"); // Change the name of the service application serviceApplication.Name = newName; // Get the database information var databaseParameters = SPDatabaseParameters.CreateParameters(dbName, dbServer, dbUserName, dbPassword, failOverServerInstance, SPDatabaseParameterOptions. None); try { serviceApplication = MCServiceUtility.UpdateServiceApplication(this, serviceApplication, databaseParameters, applicationPool, defaultEndpointName); } catch (Exception ex) { Log.ErrorFormat(LogCategory.ServiceApplication, "Unable to update service application: {0}", ex); Log.Exception(LogCategory.ServiceApplication, ex); throw; } return serviceApplication; } [SharePointPermission(SecurityAction.Demand, ObjectModel = true)] public MCServiceApplicationProxy UpdateProxy(MCServiceApplicationProxy proxy, string newProxyName, MCServiceApplication serviceApplication) { if (proxy == null) throw new ArgumentNullException("proxy"); if (string.IsNullOrEmpty(newProxyName)) throw new ArgumentNullException("newProxyName"); if (serviceApplication == null) throw new ArgumentNullException("serviceApplication"); var serviceApplicationType = serviceApplication.GetType(); if (serviceApplicationType == null || serviceApplicationType != typeof (MCServiceApplication)) throw new NotSupportedException(); proxy.Name = newProxyName; proxy.Update(); return proxy; } #endregion #region Private Methods private static bool ValidateApplicationType(Type serviceApplicationType) { if ((serviceApplicationType != null) && (serviceApplicationType == typeof (MCServiceApplication))) { return true; } Log.Warn(LogCategory.ServiceApplication, "Invalid service application type"); return false; } #endregion } }
MCServiceApplication
This class is the meat of the entire service application. Let me draw attention to various elements:
- Uses a custom MCServiceApplicationBackupBehavior attribute class (see download) which has some custom backup / restore logic built-into it (recommended over and against implementing the IBackupRestore interface), or you can just use the IisWebServiceApplicationBackupBehaviorAttribute if you’d like.
- Uses private variables decorated with the PersistedAttribute class which ensures that the variable state is maintained whenever the application is reset/recycled.
- Provides a gateway to access underlying databases leveraged by the service application, in our case a custom MCDatabase which is all provisioned and managed within this service application
- Exposes security APIs with custom permissions and access rights
- Provisions/Unprovisions a custom database for the service application
- Provisions/Unprovisions jobs running as part of the service application
- Supports all WCF endpoint protocols (http(s) and tcp)
I tried to add comments throughout the code to explain the reason for certain things, I hope the comments help.
using System; using System.Collections.Generic; using System.Linq; using System.Reflection; using System.Runtime.InteropServices; using System.Security.Permissions; using System.Security.Principal; using System.ServiceModel; using Microsoft.SharePoint; using Microsoft.SharePoint.Administration; using Microsoft.SharePoint.Administration.AccessControl; using Microsoft.SharePoint.Administration.Claims; using Microsoft.SharePoint.Security; using MyCorp.SP.ServiceApplication.Attributes; using MyCorp.SP.ServiceApplication.Jobs; using MyCorp.SP.ServiceApplication.Security; namespace MyCorp.SP.ServiceApplication { [Guid(Constants.ServiceApplicationGuid)] [MCServiceApplicationBackupBehavior] [SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true), SharePointPermission(SecurityAction.Demand, ObjectModel = true)] [ServiceBehavior(IncludeExceptionDetailInFaults = true, InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)] public class MCServiceApplication : SPIisWebServiceApplication { [Persisted] /* This attribute ensures the database is part of backup / restore, and maintained between app startups */ private MCServiceDatabase _database; [Persisted] private string _defaultEndPointName; #region Constructors public MCServiceApplication() { } internal MCServiceApplication(string name, MCService service, MCServiceDatabase database, SPIisWebServiceApplicationPool applicationPool) : base(name, service, applicationPool) { ArgumentValidator.IsNotNull(database, "database"); _database = database; Status = SPObjectStatus.Provisioning; } #endregion #region Typical Properties and Methods internal MCService MCService { get { if (!(base.Service is MCService)) { throw new InvalidOperationException(); } return (base.Service as MCService); } } public MCServiceDatabase Database { get { return _database; } private set { _database = value; } } internal string DatabaseName { get { return _database.Name; } } internal string DatabaseServer { get { return _database.ServiceInstance.NormalizedDataSource; } } internal string DatabaseConnectionString { get { return _database.DatabaseConnectionString; } } public Guid ServiceApplicationProxyId { get { return ServiceApplicationProxy.Id; } } public MCServiceApplicationProxy ServiceApplicationProxy { get { var serviceProxy = MCServiceUtility.GetLocalServiceProxy(true); var proxies = MCServiceUtility.GetApplicationProxies(serviceProxy); if (proxies != null) { return proxies.FirstOrDefault(p => p.ServiceApplicationId == Id); } return null; } } private void Install() { Log.InfoFormat(LogCategory.ServiceApplication, "Installing MyCorp Service Application '{0}'.", new object[] {base.Name}); InstallDatabase(); InstallJobs(); Log.InfoFormat(LogCategory.ServiceApplication, "MyCorp Service Application '{0}' installation complete.", new object[] {base.Name}); } private void InstallDatabase() { // provision the database so that the service // application has something to connect to Database.Provision(); EnsureDatabaseAccess(ApplicationPool.ProcessAccount.SecurityIdentifier); } private void Uninstall(bool deleteData) { Log.InfoFormat(LogCategory.ServiceApplication, "Uninstalling MyCorp Service Application '{0}'.", new object[] {base.Name}); RemoveJobs(); if (deleteData) UninstallDatabase(); Log.InfoFormat(LogCategory.ServiceApplication, "MyCorp Service Application '{0}' uninstallation complete.", new object[] {base.Name}); } private void UninstallDatabase() { if (Database != null) { Database.Unprovision(); } } #endregion #region Security Rights internal SPSiteAdministration GetCentralAdminSite() { using (SPSite site = SPAdministrationWebApplication.GetInstanceLocalToFarm(base.Farm).Sites["/"]) { return new SPSiteAdministration(site.ID); } } internal void DemandAdministrationReadAccess() { DemandAdministrationAccess(SPCentralAdministrationRights.Read); } // As opposed to checking only for the effective permissions, we want to explicity check for the given rights, because // the effective permissions are static, whereas the rights can be customized internal bool CheckAdministrationAccess(MCServiceApplicationCentralAdminRights rights) { if (base.CheckAdministrationAccess((SPCentralAdministrationRights) rights)) { return true; } //NOT THIS: var effectivePermisisons = base.GetAccessControl().ToAcl().EffectivePermissions(); var accessRules = GetAdministrationAccessControl().GetAccessRules(); foreach (var rule in accessRules) { var allowedRights = (MCServiceApplicationCentralAdminRights) rule.AllowedRights; if (allowedRights.Has(rights)) return true; } return false; } // As opposed to checking only for the effective permissions, we want to explicity check for the given rights, because // the effective permissions are static, whereas the rights can be customized internal bool CheckAccess(MCServiceApplicationRights rights) { if (base.CheckAccess((SPIisWebServiceApplicationRights) rights)) { return true; } //NOT THIS: var effectivePermisisons = base.GetAccessControl().ToAcl().EffectivePermissions(); var accessRules = GetAccessControl().GetAccessRules(); foreach (var rule in accessRules) { var allowedRights = (MCServiceApplicationRights) rule.AllowedRights; if (allowedRights.Has(rights)) return true; } return false; } private void EnsureAccess(SPProcessIdentity identity) { ArgumentValidator.IsNotNull(identity, "identity"); SecurityIdentifier currentSecurityIdentifier = identity.CurrentSecurityIdentifier; if (currentSecurityIdentifier != null) { SPIisWebServiceApplicationSecurity accessControl = GetAccessControl(); string identifier = currentSecurityIdentifier.Translate(typeof (NTAccount)).Value; SPClaim claim = SPClaimProviderManager.Local.ConvertIdentifierToClaim(identifier, SPIdentifierTypes. WindowsSamAccountName); SPAce ace = accessControl.ToAcl()[SPClaimProviderManager.Local.EncodeClaim(claim)]; if ((ace == null) || ((ace.GrantRightsMask) != ~SPIisWebServiceApplicationRights.None)) { accessControl.SetAccessRule(new SPAclAccessRule(claim, ~SPIisWebServiceApplicationRights .None)); SetAccessControl(accessControl); } } } private void EnsureDatabaseAccess(SecurityIdentifier sid) { ArgumentValidator.IsNotNull(sid, "sid"); Database.EnsureAccess(sid); } public override SPIisWebServiceApplicationSecurity GetAccessControl() { var accessControl = base.GetAccessControl(); var list = new List>(); foreach (var rule in accessControl.GetAccessRules()) { if (SPClaimProviderManager.IsEncodedClaim(rule.Name)) { var claim = SPClaimProviderManager.Local.DecodeClaim(rule.Name); if ((claim != null) && (!SPClaimTypes.Equals(claim.ClaimType, SPClaimTypes.UserLogonName) || !SPOriginalIssuers.IsIssuerType(SPOriginalIssuerType.Windows, claim.OriginalIssuer))) { list.Add(rule); } } } foreach (var rule2 in list) { accessControl.RemoveAccessRule(rule2); } return accessControl; } // Specify if there are situations where central admin application rights can override application rights (user service application uses this method) internal static MCServiceApplicationCentralAdminRights GetAdminOverrideRights( MCServiceApplicationRights applicationRights) { var none = MCServiceApplicationCentralAdminRights.None; if ((applicationRights & (MCServiceApplicationRights.None | MCServiceApplicationRights.UseUtilities)) == (MCServiceApplicationRights.None | MCServiceApplicationRights.UseUtilities)) { // Users that can Manage Utilities can always Use Utilities none |= MCServiceApplicationCentralAdminRights.None | MCServiceApplicationCentralAdminRights.ManageUtilities; } return none; } #endregion #region Typical Property and Method Overrides public override Guid ApplicationClassId { get { return new Guid(Constants.ServiceApplicationClassId); } } public override Version ApplicationVersion { get { return new Version(Constants.ServiceApplicationVersion); } } public override string TypeName { get { return Constants.ServiceApplicationTypeName; } } // InstallPath is the base path to the WCF service: WebServices\MyCorp protected override string InstallPath { get { return Constants.ServiceApplicationInstallPath; } } // VirtualPath is the actual WCF "svc" file, best to put a "dummy" name there that you can do a string replace function on // in your channel factory so that you can host multiple "svc" files instead of just one protected override string VirtualPath { get { return Constants.ServiceApplicationVirtualPath; } } // The protocol you are planning to use by default (typically: tcp or https) protected override string DefaultEndpointName { get { return _defaultEndPointName.IsNullOrEmpty() ? Constants.ServiceApplicationDefaultEndpointName : _defaultEndPointName; } } // Access Rights Customizations protected override SPNamedCentralAdministrationRights[] AdministrationAccessRights { get { return new[] { SPNamedCentralAdministrationRights.FullControl, // Admin right for managing utilities in the service application new SPNamedCentralAdministrationRights("Manage Utilities", (SPCentralAdministrationRights) (MCServiceApplicationCentralAdminRights.None | MCServiceApplicationCentralAdminRights.Read | MCServiceApplicationCentralAdminRights. ManageUtilities)), }; } } protected override SPNamedIisWebServiceApplicationRights[] AccessRights { get { return new[] { SPNamedIisWebServiceApplicationRights.FullControl, // Access right for using utilities in the service application new SPNamedIisWebServiceApplicationRights("Use Utilities", (SPIisWebServiceApplicationRights) (MCServiceApplicationRights.None | MCServiceApplicationRights.Read | MCServiceApplicationRights.UseUtilities)), }; } } // Links to UI Layout Pages to Manage Service Application in Central Admin public override SPAdministrationLink ManageLink { get { return new SPAdministrationLink(string.Format(Constants.ServiceApplicationManageLinkFormat, Id)); } } // Links to UI Layout Pages to Manage Service Application Properties in Central Admin public override SPAdministrationLink PropertiesLink { get { return new SPAdministrationLink(string.Format(Constants.ServiceApplicationUpdateLinkFormat, Id)); } } // The following property is usually not overriden (SharePoint has a good enough UI for managing service application permissions in most cases) // One thing that could be done here is do something similar to the UserProfileApplication, which is to create a custom UI and store the ACL in a // data cache with the ServiceProxy and UserProfileApplicationProxy, to avoid round-trips just to be denied due to an access rights violation // This is a little more complex as you would use the SPCache.Cache API or serialize the Acl to a string for persistence to a property field // For now, we'll leave this as is ... public override SPAdministrationLink PermissionsLink { get { return base.PermissionsLink; } } internal void SetDefaultEndpointName(string defaultEndpointName) { _defaultEndPointName = defaultEndpointName; } public override void Provision() { if (SPObjectStatus.Provisioning != base.Status) { Status = SPObjectStatus.Provisioning; Update(); } Install(); base.Provision(); Status = SPObjectStatus.Online; Update(); } public override void Unprovision(bool deleteData) { if (SPObjectStatus.Unprovisioning != base.Status) { Status = SPObjectStatus.Unprovisioning; Update(); } Uninstall(deleteData); base.Unprovision(deleteData); Status = SPObjectStatus.Disabled; Update(); } public override void Update() { DemandAdministrationAccess(SPCentralAdministrationRights.Write); bool dbCreated = false; try { if ((Database != null) && !Database.DbCreated) { Database.Update(); dbCreated = true; } base.Update(); // Support all endpoint protocols Log.Debug(LogCategory.ServiceApplication, "Adding service application endpoints"); EnsureServiceEndpoint("tcp", SPIisWebServiceBindingType.NetTcp, null); EnsureServiceEndpoint("tcp-ssl", SPIisWebServiceBindingType.NetTcp, "secure"); EnsureServiceEndpoint("http", SPIisWebServiceBindingType.Http, null); EnsureServiceEndpoint("https", SPIisWebServiceBindingType.Https, "secure"); base.Update(); } catch { if (dbCreated) { try { Database.Delete(); } catch (Exception exception) { Log.ErrorFormat(LogCategory.ServiceApplication, "Error deleting MyCorp Service Application Database: {0}", new object[] {exception.ToString()}); } } throw; } } public override void Delete() { // Delete related jobs RemoveJobs(); // Delete the service application // This must be done BEFORE the database is deleted, or else a dependency error will occur base.Delete(); if (Database != null) { // IF there are other service applications that have a dependency on this database, // you cannot delete the database object (only the last dependency can delete it) // This does not delete the physical database, only the persisted object reference // to the database (Unprovision is what deletes the physical database) if (!OtherDependenciesOnDatabaseExist()) Database.Delete(); } } private bool OtherDependenciesOnDatabaseExist() { MCService service = MCServiceUtility.GetLocalService(false); if (_database != null && service != null) { IEnumerable applications = service.Applications == null ? null : MCServiceUtility.GetApplications(service); if (applications != null) { return applications.Any(a => a.Id != Id && a.Database != null && a.Database.Id == _database.Id); } } return false; } private void EnsureServiceEndpoint(string name, SPIisWebServiceBindingType bindingType, string relativeAddress) { foreach (SPIisWebServiceEndpoint endpoint in Endpoints) { if (endpoint.Name == name) return; } AddServiceEndpoint(name, bindingType, relativeAddress); } protected override void OnDependentProcessIdentityChanged() { SPIisWebServiceApplicationSecurity accessControl = GetAccessControl(); SPAcl acl = accessControl.ToAcl(); bool flag = false; foreach (SecurityIdentifier identifier in GetDependentProcessIdentities()) { string str = identifier.Translate(typeof (NTAccount)).Value; SPClaim claim = SPClaimProviderManager.Local.ConvertIdentifierToClaim(str, SPIdentifierTypes. WindowsSamAccountName); SPAce ace = acl[SPClaimProviderManager.Local.EncodeClaim(claim)]; if ((ace == null) || ((ace.GrantRightsMask) != ~SPIisWebServiceApplicationRights.None)) { accessControl.SetAccessRule(new SPAclAccessRule(claim, ~SPIisWebServiceApplicationRights .None)); flag = true; } } if (flag) { SetAccessControl(accessControl); } base.OnDependentProcessIdentityChanged(); } protected override void OnProcessIdentityChanged(SecurityIdentifier processSecurityIdentifier) { EnsureDatabaseAccess(processSecurityIdentifier); base.OnProcessIdentityChanged(processSecurityIdentifier); } #endregion #region Jobs Methods // Service Applications are a great place to provision jobs to use within the SharePoint Farm private IEnumerable JobDefinitions { get { foreach (SPJobDefinition job in Service.JobDefinitions) { var iteratorJob = job as MCServiceApplicationJobDefinition; if ((iteratorJob != null) && (iteratorJob.ServiceApplicationId == Id)) { yield return iteratorJob; } } } } internal void StopJobs() { foreach (MCServiceApplicationJobDefinition job in JobDefinitions) { if ((job != null)) { job.IsDisabled = true; job.Update(); break; } } } internal void StartJobs() { foreach (MCServiceApplicationJobDefinition job in JobDefinitions) { if ((job != null)) { job.IsDisabled = false; job.Update(); break; } } } // This method assumes that each service application job definition has a constructor with a // single parameter MCServiceApplication internal void InstallJobs() { EnsureAccess(base.Farm.TimerService.ProcessIdentity); foreach (Type type in MCServiceApplicationJobDefinition.Types) { if (!JobExists(type)) { Log.InfoFormat(LogCategory.ServiceApplication, "Installing scheduled job '{0}' for app '{1}'.", new object[] {type, base.Name}); var types = new[] {typeof (MCServiceApplication)}; var parameters = new object[] {this}; var info = type.GetConstructor(BindingFlags.NonPublic | BindingFlags.Instance, null, types, null); if (info != null) { var job = info.Invoke(parameters) as MCServiceApplicationJobDefinition; if (job != null) { job.Schedule = job.DefaultSchedule; job.Update(); Log.InfoFormat(LogCategory.ServiceApplication, "Finished installing scheduled job '{0}' for app '{1}'.", new object[] {type, base.Name}); } } } } } internal void RemoveJobs() { foreach (MCServiceApplicationJobDefinition job in JobDefinitions) { job.Delete(); } } internal bool JobExists(Type type) { foreach (MCServiceApplicationJobDefinition job in JobDefinitions) { if (type == job.GetType()) { return true; } } return false; } internal T GetJob() where T : MCServiceApplicationJobDefinition { foreach (MCServiceApplicationJobDefinition job in JobDefinitions) { var local = job as T; if (local != null) { return local; } } return default(T); } #endregion } }
MCServiceApplicationProxy
This class is crucial as it represents the means for communicating with and accessing service application functionality. Here are some important elements built-in to the class below:
- A custom class attribute MCServiceApplicationProxyBackupBehavior to extend Backup/Restore functionality (similar to the MCServiceApplication class above)
- Leverages built-in SharePoint load-balancing capabilities by using the SPServiceLoadBalancer class to open/close WCF channels
- Provides an example of how to store custom properties on the proxy that can be used internally for example for setting proxy/WCF communication parameters
- Provisions/Unprovisions jobs that may need to be part of the proxy (not very common, I only know of the User Profile Service Application that does this)
- Re-attempts WCF method communication for X number of failures
Some developers will build business logic into the SPServiceApplication and SPServiceApplicationProxy classes. I prefer not to. I like to put the business logic in separate service and manager classes, I’ll show that later in this post.
using System; using System.Collections.Generic; using System.Configuration; using System.Globalization; using System.Reflection; using System.Runtime.InteropServices; using System.Security.Permissions; using System.Security.Principal; using System.ServiceModel; using System.ServiceModel.Configuration; using System.Web; using Microsoft.IdentityModel.Claims; using Microsoft.SharePoint; using Microsoft.SharePoint.Administration; using Microsoft.SharePoint.Security; using Microsoft.SharePoint.Utilities; using MyCorp.SP.ServiceApplication.Attributes; using MyCorp.SP.ServiceApplication.Jobs; namespace MyCorp.SP.ServiceApplication { [Guid(Constants.ServiceApplicationProxyGuid)] [MCServiceApplicationProxyBackupBehavior] [SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true), SharePointPermission(SecurityAction.Demand, ObjectModel = true)] [ServiceBehavior(IncludeExceptionDetailInFaults = true, InstanceContextMode = InstanceContextMode.PerSession, ConcurrencyMode = ConcurrencyMode.Multiple)] [AspNetHostingPermission(SecurityAction.LinkDemand, Level = AspNetHostingPermissionLevel.Minimal)] public class MCServiceApplicationProxy : SPIisWebServiceApplicationProxy { [Persisted] private readonly SPServiceLoadBalancer _loadBalancer; [Persisted] private readonly Dictionary _proxySettings; [Persisted] private Guid _serviceApplicationId = Guid.Empty; #region Constructors public MCServiceApplicationProxy() { _proxySettings = new Dictionary(); InitializeProxySettings(); } public MCServiceApplicationProxy(string name, MCServiceProxy serviceProxy, MCServiceApplication serviceApplication) : base(name, serviceProxy, serviceApplication.Uri) { _serviceApplicationId = serviceApplication.Id; _loadBalancer = new SPRoundRobinServiceLoadBalancer(serviceApplication.Uri); _proxySettings = new Dictionary(); InitializeProxySettings(); base.Status = SPObjectStatus.Provisioning; } private void InitializeProxySettings() { OpenTimeout = new TimeSpan(0, 0, 60); SendTimeout = new TimeSpan(0, 0, 60); ReceiveTimeout = new TimeSpan(0, 0, 60); CloseTimeout = new TimeSpan(0, 0, 30); MaximumExecutionTime = 300; MaximumNumberOfAttemptBeforeFailing = 3; FailBalancerOnTimeout = true; } #endregion #region Typical Properties and Methods public SPServiceLoadBalancer LoadBalancer { get { return _loadBalancer; } } public Guid ServiceApplicationId { get { return _serviceApplicationId; } internal set { _serviceApplicationId = value; } } public MCServiceApplication ServiceApplication { get { var service = MCServiceUtility.GetLocalService(true); return MCServiceUtility.GetApplicationById(service, ServiceApplicationId); } } internal SPServiceApplicationProxyGroup ServiceApplicationProxyGroup { get { if (ServiceApplication != null) return ServiceApplication.ServiceApplicationProxyGroup; var group = SPServiceApplicationProxyGroup.Default; if (!group.Contains(this)) { foreach (var proxyGroup in SPFarm.Local.ServiceApplicationProxyGroups) { if (proxyGroup.Contains(this)) { return proxyGroup; } } } return group; } } private void Install() { _loadBalancer.Provision(); InstallJobs(); } private void Uninstall() { _loadBalancer.Unprovision(); RemoveJobs(); } // Provide an easy way for the WCF client to create a channel factory internal ChannelFactory CreateChannelFactory(string endpointConfigurationName) { // get client configuration from 'client.config' var configuration = OpenClientConfiguration( SPUtility.GetGenericSetupPath(Constants.ServiceApplicationProxyInstallPath)); var factory = new ConfigurationChannelFactory(endpointConfigurationName, configuration, null); factory.ConfigureCredentials(SPServiceAuthenticationMode.Claims); return factory; } #endregion #region Typical Property and Method Overrides public override string TypeName { get { return Constants.ServiceApplicationProxyTypeName; } } // Links to UI Layout Pages to Manage Service Application in Central Admin // This is OPTIONAL obviously, you may not want to have any management customizations for your application proxy, in which case // just comment this out ... public override SPAdministrationLink ManageLink { get { return new SPAdministrationLink(string.Format(Constants.ServiceApplicationProxyManageLinkFormat, Id)); } } // Links to UI Layout Pages to Manage Service Application Properties in Central Admin // This is OPTIONAL obviously, you may not want to have any properties customizations for your application proxy, in which case // just comment this out ... public override SPAdministrationLink PropertiesLink { get { return new SPAdministrationLink(string.Format(Constants.ServiceApplicationProxyPropertiesLinkFormat, Id)); } } public override void Provision() { if (SPObjectStatus.Provisioning != base.Status) { base.Status = SPObjectStatus.Provisioning; Update(); } Install(); base.Provision(); base.Status = SPObjectStatus.Online; Update(); } public override void Unprovision(bool deleteData) { if (SPObjectStatus.Unprovisioning != base.Status) { base.Status = SPObjectStatus.Unprovisioning; Update(); } Uninstall(); base.Unprovision(deleteData); base.Status = SPObjectStatus.Disabled; Update(); } #endregion #region Security Rights internal static bool IsUserAnonymous { get { var current = HttpContext.Current; if (current != null) { var identity = current.User.Identity as WindowsIdentity; if (identity != null) { return identity.IsAnonymous; } var identity2 = current.User.Identity as IClaimsIdentity; if (identity2 != null) { return !identity2.IsAuthenticated; } } return false; } } #endregion #region Jobs Methods private IEnumerable JobDefinitions { get { foreach (var job in MCServiceApplicationProxyJobDefinition.ParentService.JobDefinitions) { var iteratorJob = job as MCServiceApplicationProxyJobDefinition; if ((iteratorJob != null) && (iteratorJob.ServiceApplicationProxyId == Id)) { yield return iteratorJob; } } } } internal void StopJobs() { foreach (MCServiceApplicationProxyJobDefinition job in JobDefinitions) { if ((job != null)) { job.IsDisabled = true; job.Update(); break; } } } internal void StartJobs() { foreach (MCServiceApplicationProxyJobDefinition job in JobDefinitions) { if ((job != null)) { job.IsDisabled = false; job.Update(); break; } } } internal bool JobExists(Type type) { foreach (MCServiceApplicationProxyJobDefinition job in JobDefinitions) { if (type == job.GetType()) { return true; } } return false; } internal T GetJob() where T : MCServiceApplicationProxyJobDefinition { foreach (MCServiceApplicationProxyJobDefinition job in JobDefinitions) { var local = job as T; if (local != null) { return local; } } return default(T); } // This method assumes that each service application proxy job definition has a constructor with a // single parameter MCServiceApplicationProxy internal void InstallJobs() { foreach (Type type in MCServiceApplicationProxyJobDefinition.Types) { if (!JobExists(type)) { Log.InfoFormat(LogCategory.ServiceApplication, "Installing scheduled job '{0}' for app proxy '{1}'.", new object[] {type, base.Name}); var types = new[] {typeof (MCServiceApplicationProxy)}; var parameters = new object[] {this}; ConstructorInfo info = type.GetConstructor(BindingFlags.NonPublic | BindingFlags.Instance, null, types, null); if (info != null) { var job = info.Invoke(parameters) as MCServiceApplicationProxyJobDefinition; job.Schedule = job.DefaultSchedule; job.Update(); Log.InfoFormat(LogCategory.ServiceApplication, "Finished installing scheduled job '{0}' for app proxy '{1}'.", new object[] {type, base.Name}); } } } } private void RemoveJobs() { foreach (MCServiceApplicationProxyJobDefinition job in JobDefinitions) { job.Delete(); } } #endregion #region Custom Properties public Dictionary ProxySettings { get { return _proxySettings; } } public TimeSpan OpenTimeout { get { double timeout; return (ProxySettings.ContainsKey("ProxyOpenTimeoutInSeconds") && double.TryParse(ProxySettings["ProxyOpenTimeoutInSeconds"], out timeout)) ? TimeSpan.FromSeconds(timeout) : new TimeSpan(0, 0, 0, 60, 0); } set { ProxySettings["ProxyOpenTimeoutInSeconds"] = value.TotalSeconds.ToString(CultureInfo.InvariantCulture); } } public TimeSpan SendTimeout { get { double timeout; return (ProxySettings.ContainsKey("ProxySendTimeoutInSeconds") && double.TryParse(ProxySettings["ProxySendTimeoutInSeconds"], out timeout)) ? TimeSpan.FromSeconds(timeout) : new TimeSpan(0, 0, 0, 60, 0); } set { ProxySettings["ProxySendTimeoutInSeconds"] = value.TotalSeconds.ToString(CultureInfo.InvariantCulture); } } public TimeSpan ReceiveTimeout { get { double timeout; return (ProxySettings.ContainsKey("ProxyReceiveTimeoutInSeconds") && double.TryParse(ProxySettings["ProxyReceiveTimeoutInSeconds"], out timeout)) ? TimeSpan.FromSeconds(timeout) : new TimeSpan(0, 0, 0, 60, 0); } set { ProxySettings["ProxyReceiveTimeoutInSeconds"] = value.TotalSeconds.ToString(CultureInfo.InvariantCulture); } } public TimeSpan CloseTimeout { get { double timeout; return (ProxySettings.ContainsKey("ProxyCloseTimeoutInSeconds") && double.TryParse(ProxySettings["ProxyCloseTimeoutInSeconds"], out timeout)) ? TimeSpan.FromSeconds(timeout) : new TimeSpan(0, 0, 0, 30, 0); } set { ProxySettings["ProxyCloseTimeoutInSeconds"] = value.TotalSeconds.ToString(CultureInfo.InvariantCulture); } } public uint MaximumExecutionTime { get { uint maxTime; return (ProxySettings.ContainsKey("ProxyCallMaximumExecutionTimeInSeconds") && uint.TryParse(ProxySettings["ProxyCallMaximumExecutionTimeInSeconds"], out maxTime)) ? maxTime : 300; } set { ProxySettings["ProxyCallMaximumExecutionTimeInSeconds"] = value.ToString(CultureInfo.InvariantCulture); } } public uint MaximumNumberOfAttemptBeforeFailing { get { uint numAttempts; return (ProxySettings.ContainsKey("ProxyCallMaxNumberOfAttemptsBeforeFailing") && uint.TryParse(ProxySettings["ProxyCallMaxNumberOfAttemptsBeforeFailing"], out numAttempts)) ? numAttempts : 5; } set { ProxySettings["ProxyCallMaxNumberOfAttemptsBeforeFailing"] = value.ToString(CultureInfo.InvariantCulture); } } public bool FailBalancerOnTimeout { get { bool failOnTimeout = true; if (ProxySettings.ContainsKey("ProxyFailBalancerOnTimeout")) failOnTimeout = ProxySettings["ProxyFailBalancerOnTimeout"].NullSafe().ToLowerInvariant() == bool.TrueString.ToLowerInvariant(); return failOnTimeout; } set { ProxySettings["ProxyFailBalancerOnTimeout"] = value.ToString(CultureInfo.InvariantCulture); } } #endregion } }
MCServiceInstance, MCServiceProxy
These classes are also very important for the correct functioning of the service application (I describe them in the last post of this series). The code for these classes is not really complex, so take a look at the download for more details on these.
MCServiceUtility
This is a custom class I created that simply has generic methods to easily access the primary service application classes that may be running within the farm. Again, take a look at the download for more details on this one.
MCDatabase
Now this is a pretty cool class IMO. This represents a custom database that we want to include as part of our service application. Service applications don’t have to have databases, but having one can really be useful as you might imagine, because you can store any information you’d like in it, scale it however you’d like, mirror it and let SharePoint handle dispatching requests to the appropriate database as needed (for failover purposes). The way I like to structure the database implementation in a service application is by leveraging SQL scripts deployed to the hive (and keeping track of the scripts that have been executed in a custom DbScripts table in the database). This way, whenever you want to add functionality to your service application (as-in when you are building a new custom app within the service application, or adding some extension points for yet another 3rd party system), you can just drop SQL scripts into the hive as part of your deployment, and the service application will automatically read them in for you.
Let’s take a look at the class below. Notice the following:
- Uses Dapper as a utility library for executing SQL
- Provisions the database as part of the service application when first creating the service application
- Checks a SQL directory for new scripts to run on Create and on Update –> In my implementation the first script in the directory should have the CREATE TABLE DbScripts SQL in it. The code download is already setup for you to install and run with this implementation.
- Automatically grants the application pool access to the database
- Provides a clean way for managing database upgrades across service application revisions/deployments
using System; using System.Collections; using System.Collections.Generic; using System.Collections.Specialized; using System.Data; using System.Data.SqlClient; using System.IO; using System.Linq; using System.Runtime.InteropServices; using System.Security.Permissions; using System.Security.Principal; using System.Text; using System.Web; using Dapper; using Microsoft.SharePoint.Administration; using Microsoft.SharePoint.Security; using Microsoft.SharePoint.Upgrade; using Microsoft.SharePoint.Utilities; namespace MyCorp.SP.ServiceApplication { [Guid(Constants.ServiceDatabaseGuid)] [SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true), SharePointPermission(SecurityAction.InheritanceDemand, ObjectModel = true)] [AspNetHostingPermission(SecurityAction.LinkDemand, Level = AspNetHostingPermissionLevel.Minimal)] public class MCServiceDatabase : SPDatabase { #region Constructors public MCServiceDatabase() { base.Status = SPObjectStatus.Offline; } public MCServiceDatabase(SPDatabaseParameters databaseParameters) : base(databaseParameters) { base.Status = SPObjectStatus.Offline; } #endregion #region Typical Properties and Methods public bool DbCreated { get { return base.WasCreated; } } internal static bool IsDatabaseValid(string databaseName, string databaseServer, out string errorMessage) { errorMessage = string.Empty; foreach (MCServiceApplication application in MCServiceUtility.GetApplications()) { if ((!string.IsNullOrEmpty(application.DatabaseName) && !string.IsNullOrEmpty(application.DatabaseServer)) && (application.DatabaseName.Equals(databaseName, StringComparison.OrdinalIgnoreCase) && application.DatabaseServer.Equals(databaseServer, StringComparison.OrdinalIgnoreCase))) { errorMessage = string.Format("Database is already in use by the '{0}' service application", new object[] {application.Name}); return false; } } return true; } internal void EnsureAccess(SecurityIdentifier sid) { GrantApplicationPoolAccess(sid); GrantAllApplicationPoolsAccess(); // this just makes it easier 🙂 } internal void GrantApplicationPoolAccess(SecurityIdentifier sid) { GrantAccess(sid, "db_owner"); } internal void GrantAllApplicationPoolsAccess() { GrantApplicationPoolAccess(SPFarm.Local.TimerService.ProcessIdentity.ManagedAccount.Sid); foreach (var pool in SPWebService.AdministrationService.ApplicationPools) { try { GrantApplicationPoolAccess(pool.ManagedAccount.Sid); } catch (Exception ex) { Log.ErrorFormat(LogCategory.ServiceApplication, "Unable to grant access to database {0} to user {1}.", Name, pool.Username); Log.Exception(LogCategory.ServiceApplication, ex); } } } #endregion #region Typical Property and Method Overrides public override string TypeName { get { return Constants.ServiceDatabaseTypeName; } } // Check to see if the schema needs to be upgraded or not // In our case, with this implementation only NEW scripts are executed, so we // can let the process attempt the upgrade every time (it will just ignore any scripts that have // already been executed) private bool ContainsValidSchema() { return true; } public override void Provision() { if (SPObjectStatus.Online == Status) { Log.InfoFormat(LogCategory.ServiceApplication, "MCServiceDatabase '{0}' provisioning aborted, database is already online.", new object[] {base.Name}); return; } Log.InfoFormat(LogCategory.ServiceApplication, "MCServiceDatabase '{0}' provisioning started.", new object[] {base.Name}); Status = SPObjectStatus.Provisioning; var options = new Dictionary {{SqlDatabaseOption[(int) DatabaseOptions.AutoClose], false}}; Update(); if (!base.Exists || base.IsEmpty()) { Log.Info(LogCategory.ServiceApplication, "Processing MyCorp Service Database Upgrade Scripts"); // get the first file in the directory as the provisioning file var directoryPath = SPUtility.GetGenericSetupPath(Constants.ServiceApplicationSqlDirectory); var scriptPaths = Directory.GetFiles(directoryPath, "*.sql").Select(d => d.ToLower()).OrderBy(d => d); if (scriptPaths.Any()) { Provision(scriptPaths.First(), options); } NeedsUpgrade = false; } if (ContainsValidSchema()) { try { NeedsUpgrade = true; if (!SPManager.PeekIsUpgradeRunning()) { SPManager.Instance.RunUpgradeSession(this, false); Upgrade(); } else { base.Upgrade(); } } catch (SPUpgradeException ex) { Log.WarnFormat(LogCategory.ServiceApplication, "Unable to successfully upgrade MyCorp Service Database: {0}", ex); } base.GrantOwnerAccessToDatabaseAccount(); Status = SPObjectStatus.Online; Update(); Log.InfoFormat(LogCategory.ServiceApplication, "MCServiceDatabase '{0}' provisioning complete.", new object[] {base.Name}); } else { throw new InvalidOperationException("Invalid MyCorp Service Database schema"); } } private void Provision(string scriptPath, Dictionary options) { Provision(DatabaseConnectionString, scriptPath, options); string scriptName = new FileInfo(scriptPath).Name.ToLower().Trim(); using (var connection = new SqlConnection(DatabaseConnectionString)) { connection.Open(); int count = connection.Query("SELECT COUNT(ID) FROM DbScripts WHERE [ScriptName] = @ScriptName", new {ScriptName = scriptName.ToLower()}).Single(); if (count == 0) { connection.Execute("INSERT INTO DbScripts ( [ScriptName] ) VALUES ( @ScriptName )", new {ScriptName = scriptName}); } } } public override void Unprovision() { Log.InfoFormat(LogCategory.ServiceApplication, "MCServiceDatabase '{0}' unprovisioning started.", new object[] {base.Name}); base.Status = SPObjectStatus.Unprovisioning; Update(); base.Unprovision(); base.Status = SPObjectStatus.Offline; Update(); Log.InfoFormat(LogCategory.ServiceApplication, "MCServiceDatabase '{0}' unprovisioning complete.", new object[] {base.Name}); } public override void Upgrade() { Log.Info(LogCategory.ServiceApplication, "Upgrading the MyCorp Service Database"); // Execute any upgrade scripts desired here var builder = new SqlConnectionStringBuilder(DatabaseConnectionString) { Pooling = false }; var connectionString = builder.ToString(); var minimumCommandTimeout = 0; if (SPFarm.Local != null) { var service = SPFarm.Local.Services.GetValue(); minimumCommandTimeout = service.CommandTimeout; } // Retrieve all upgrade scripts var directoryPath = SPUtility.GetGenericSetupPath(Constants.ServiceApplicationSqlDirectory); var scripts = new OrderedDictionary(); if (Directory.Exists(directoryPath)) { // skip the first file IEnumerable scriptPaths = Directory.GetFiles(directoryPath, "*.sql").Select(d => d.ToLower()).OrderBy(d => d).Skip(1); foreach (string scriptPath in scriptPaths) { string fileName = Path.GetFileName(scriptPath); if (!string.IsNullOrEmpty(fileName)) scripts.Add(fileName.ToLower(), scriptPath); } } Log.Info(LogCategory.ServiceApplication, "Attempting to upgrade the database with any available SQL scripts"); SqlConnection sqlConnection = null; SqlTransaction sqlTransaction = null; try { sqlConnection = new SqlConnection(connectionString); sqlConnection.Open(); Log.Debug(LogCategory.ServiceApplication, "Checking for new upgrade scripts to run"); // Determine which scripts have not yet been run IEnumerable executedScripts = sqlConnection.Query("SELECT [ScriptName] FROM DbScripts ORDER BY [ScriptName]", null); var upgradeScripts = scripts.Cast().Where( x => !executedScripts.Contains(x.Key.ToString().ToLower())).Select( x => new {Key = x.Key.ToString(), Value = x.Value.ToString()}).ToList(); if (upgradeScripts.Any()) { sqlTransaction = sqlConnection.BeginTransaction(IsolationLevel.ReadUncommitted, "DbScripts"); foreach (var upgradeScript in upgradeScripts.OrderBy(x => x.Key)) { Log.InfoFormat(LogCategory.ServiceApplication, "Processing new SQL script: " + upgradeScript); var commands = ParseCommands(upgradeScript.Value, false); foreach (string commandText in commands) { sqlConnection.Execute(commandText, null, sqlTransaction, minimumCommandTimeout, CommandType.Text); } sqlConnection.Execute("INSERT INTO DbScripts ( [ScriptName] ) VALUES ( @ScriptName )", new {ScriptName = upgradeScript.Key}, sqlTransaction, minimumCommandTimeout, CommandType.Text); } sqlTransaction.Commit(); } Log.Info(LogCategory.ServiceApplication, "All available SQL scripts successfully processed"); } catch (Exception ex) { if (sqlTransaction != null) sqlTransaction.Rollback(); Log.ErrorFormat(LogCategory.ServiceApplication, "Unable to upgrade database, transactions rolled back: {0}", ex.Message); Log.Exception(LogCategory.ServiceApplication, ex); if (ex is SqlException) { var sex = ex as SqlException; var sqlErrors = new StringBuilder(); for (int i = 0; i < sex.Errors.Count; i++) { sqlErrors.AppendFormat( "Index #{0}\n\tMessage: {1}\n\tLineNumber: {2}\n\tSource: {3}\n\tProcedure: {4}\n", i, sex.Errors[i].Message, sex.Errors[i].LineNumber, sex.Errors[i].Source, sex.Errors[i].Procedure); } Log.ErrorFormat(LogCategory.ServiceApplication, "SqlErrors: \n{0}", sqlErrors); Log.Exception(LogCategory.ServiceApplication, ex); } throw; } finally { if (sqlTransaction != null) sqlTransaction.Dispose(); if (sqlConnection != null) { if (sqlConnection.State != ConnectionState.Closed) sqlConnection.Close(); sqlConnection.Dispose(); } } Log.Info(LogCategory.ServiceApplication, "Finished upgrading the MyCorp Service Database"); } #endregion #region Helpers private static IEnumerable ParseCommands(string filePath, bool throwExceptionIfNonExists) { if (!File.Exists(filePath)) { if (throwExceptionIfNonExists) throw new FileNotFoundException("File not found", filePath); else return new string[0]; } Log.DebugFormat(LogCategory.ServiceApplication, "Parsing SQL script file: {0}", filePath); var statements = new List(); using (var stream = File.OpenRead(filePath)) using (var reader = new StreamReader(stream)) { string statement = ""; while ((statement = ReadNextStatementFromStream(reader)) != null) { statements.Add(statement); } } return statements.ToArray(); } private static string ReadNextStatementFromStream(TextReader reader) { var sb = new StringBuilder(); while (true) { string lineOfText = reader.ReadLine(); if (lineOfText == null) { if (sb.Length > 0) return sb.ToString(); else return null; } if (lineOfText.TrimEnd().ToUpper() == "GO") break; sb.Append(lineOfText + Environment.NewLine); } return sb.ToString(); } #endregion } }
MCServiceClient (abstract class)
This class does all the heavy lifting for managing the WCF communication. Here are some features of this class:
- Caches channel factories as they are expensive to create (tries to re-use them as much as possible)
- Allows for many endpoint addresses by doing a simple string.Replace on a phantom svc endpoint address (this allows us to have many endpoints within the service application and grow the service application as big as we need to). Each endpoint can have different binding information in the configuration as needed.
- Executes code blocks in the form of delegates, as this allows us to have a generic implementation for calling all service methods from client code
- Handles exceptions by adhering to various conventions: FaultExceptions fail immediately (as do application exceptions and security exceptions), other failures result in the proxy to re-attempt the call X number of times (setting on the Proxy class)
- Efficiently aborts the WCF communication if an error occurs
using System; using System.Collections.Generic; using System.ServiceModel; using System.ServiceModel.Channels; using System.ServiceModel.Security; using System.Threading; using Microsoft.SharePoint; using Microsoft.SharePoint.Administration; using Microsoft.SharePoint.Utilities; using MyCorp.SP.ServiceApplication.ServiceModel; namespace MyCorp.SP.ServiceApplication.ServiceClients { /// /// The base client class to interoperate with WCF methods /// /// The service interface that defines the methods and operations available for the client to call public abstract class MCServiceClient { // Re-use channel factories as much as possible private static readonly Dictionary> _channelFactories = new Dictionary>(); private static object _channelFactoryLock = new object(); private readonly SPServiceContext _serviceContext; private MCServiceApplicationProxy _applicationProxy; #region Constructors protected MCServiceClient() : this(null) { } protected MCServiceClient(SPServiceContext serviceContext) { if (serviceContext == null) serviceContext = SPServiceContext.Current ?? SPServiceContext.GetContext(SPServiceApplicationProxyGroup.Default, new SPSiteSubscriptionIdentifier(Guid.Empty)); if (serviceContext == null) throw new ArgumentNullException("serviceContext"); _serviceContext = serviceContext; InitializeProxy( serviceContext.GetDefaultProxy(typeof (MCServiceApplicationProxy)) as MCServiceApplicationProxy); } private void InitializeProxy(MCServiceApplicationProxy proxy) { _applicationProxy = proxy; if (_applicationProxy == null) throw new InvalidOperationException("Service application proxy is null"); // get default timeouts defined by proxy OpenTimeout = _applicationProxy.OpenTimeout; SendTimeout = _applicationProxy.SendTimeout; ReceiveTimeout = _applicationProxy.ReceiveTimeout; CloseTimeout = _applicationProxy.CloseTimeout; MaximumExecutionTime = _applicationProxy.MaximumExecutionTime; MaximumNumberOfAttemptBeforeFailing = _applicationProxy.MaximumNumberOfAttemptBeforeFailing; FailBalancerOnTimeout = _applicationProxy.FailBalancerOnTimeout; } #endregion #region Channel and ChannelFactory Methods private TService GetChannel(SPServiceLoadBalancerContext context, bool asLoggedOnUser) { // get the channel factory var channelFactory = GetOrCreateChannelFactory(GetEndpointName(context.EndpointAddress)); // Use string.Replace to substitute the dummy svc pointer with the actual svc file the TService implementation requires var endpointUri = context.EndpointAddress.AbsoluteUri.Replace(Constants.ServiceApplicationVirtualPath, EndpointSvcFile); var endpointAddress = new EndpointAddress(new Uri(endpointUri), new AddressHeader[0]); TService serviceContract; if (asLoggedOnUser) { using (new SPMonitoredScope("ChannelFactory.CreateChannelActingAsLoggedOnUser")) { serviceContract = channelFactory.CreateChannelActingAsLoggedOnUser(endpointAddress); } } else { serviceContract = channelFactory.CreateChannelAsProcess(endpointAddress); } if (serviceContract is ICommunicationObject) ((ICommunicationObject) serviceContract).Open(); return serviceContract; } private ChannelFactory GetOrCreateChannelFactory(string endpointConfigurationName) { var key = endpointConfigurationName + Proxy.Id.ToString("N"); // Check for a cached channel factory for the endpoint configuration ChannelFactory channelFactory; if (!_channelFactories.TryGetValue(key, out channelFactory)) { lock (_channelFactoryLock) { // Double check to be sure the channel factory will not be created twice if (!_channelFactories.TryGetValue(key, out channelFactory)) { channelFactory = Proxy.CreateChannelFactory(endpointConfigurationName); _channelFactories.Add(key, channelFactory); } } } // Set the channel factory timeouts channelFactory.Endpoint.Binding.OpenTimeout = OpenTimeout; channelFactory.Endpoint.Binding.SendTimeout = SendTimeout; channelFactory.Endpoint.Binding.ReceiveTimeout = ReceiveTimeout; channelFactory.Endpoint.Binding.CloseTimeout = CloseTimeout; return channelFactory; } protected virtual void ExecuteOnChannel(Action codeBlock, bool asLoggedOnUser) { ArgumentValidator.IsNotNull(codeBlock, "codeBlock"); if (Proxy.Status != SPObjectStatus.Online) throw new InvalidOperationException("MyCorp Service Application Proxy is not online"); var operationName = codeBlock.Method.Name; var operationDescription = string.Format("ExecuteOnChannel {0}: {1}", GetType().Name, operationName); Log.DebugFormat(LogCategory.ServiceApplication, "{0}", operationDescription); try { using ( new SPMonitoredScope(operationDescription, MaximumExecutionTime, new ISPScopedPerformanceMonitor[] {new SPExecutionTimeCounter(MaximumExecutionTime)})) { ExecuteCodeBlock(codeBlock, asLoggedOnUser); } } catch (FaultException serviceFaultException) { Log.ErrorFormat(LogCategory.ServiceApplication, "FaultException: {0} ({1})\nSource:{2}\n Message:{3}\n Exception:{4}", operationDescription, serviceFaultException.Message, serviceFaultException.Detail.Source.NullSafe(), serviceFaultException.Detail.Message.NullSafe(), serviceFaultException.Detail.Exception.NullSafe()); Log.Exception(LogCategory.ServiceApplication, serviceFaultException); throw; } catch (FaultException faultException) { Log.ErrorFormat(LogCategory.ServiceApplication, "FaultException {0} ({1}): {2}: {3}", new object[] { operationDescription, faultException.Message, faultException.GetType().Name, faultException.StackTrace }); Log.Exception(LogCategory.ServiceApplication, faultException); throw; } catch (Exception exception) { Log.ErrorFormat(LogCategory.ServiceApplication, "{2}: {0} ({1}) - User {3} ({4})", operationDescription, exception.Message, exception.GetType().Name, exception.StackTrace, Thread.CurrentPrincipal.Identity.Name, Thread.CurrentPrincipal.Identity.AuthenticationType); Log.Exception(LogCategory.ServiceApplication, exception); throw; } } private void ExecuteCodeBlock(Action codeBlock, bool asLoggedOnUser) { ArgumentValidator.IsNotNull(codeBlock, "codeBlock"); if (Proxy.Status != SPObjectStatus.Online) throw new EndpointNotFoundException("VCP Service Application Proxy is not online"); var loadBalancer = Proxy.LoadBalancer; if (loadBalancer == null) throw new InvalidOperationException("Load Balancer not found."); var executing = true; uint numberOfAttempts = 0; while (executing && (numberOfAttempts < MaximumNumberOfAttemptBeforeFailing)) { numberOfAttempts++; try { var context = loadBalancer.BeginOperation(); try { using (new SPServiceContextScope(_serviceContext)) { var channel = (ICommunicationObject) GetChannel(context, asLoggedOnUser); try { codeBlock((TService) channel); executing = false; // code is done executing channel.Close(); } finally { if (channel.State != CommunicationState.Closed) { channel.Abort(); } } } } catch (TimeoutException) { if (FailBalancerOnTimeout) { context.Status = SPServiceLoadBalancerStatus.Failed; } throw; } catch (EndpointNotFoundException) { context.Status = SPServiceLoadBalancerStatus.Failed; throw; } catch (ServerTooBusyException) { context.Status = SPServiceLoadBalancerStatus.Failed; throw; } catch (CommunicationObjectFaultedException) { context.Status = SPServiceLoadBalancerStatus.Failed; throw; } catch (CommunicationException exception) { if (!(exception is FaultException)) { context.Status = SPServiceLoadBalancerStatus.Failed; } throw; } finally { context.EndOperation(); } } catch (SecurityAccessDeniedException) { // fail on security access denied throw; } catch (MCServiceSecurityException) { // fail on application security exception throw; } catch (MCServiceException) { // fail on application exception throw; } catch (FaultException) { // fail on fault exception throw; } catch (Exception ex) { if (numberOfAttempts == MaximumNumberOfAttemptBeforeFailing) Log.WarnFormat(LogCategory.ServiceApplication, "Attempted WCF operation {0} ({1}) {2} times before failing.", GetType().Name, codeBlock.Method.Name, numberOfAttempts); Log.Exception(LogCategory.ServiceApplication, ex); // if the maximum number of attempts hasn't been reached, retry the execution if (!executing) throw; } } } #endregion #region Typical Properties public MCServiceApplicationProxy Proxy { get { return _applicationProxy; } } public abstract string EndpointSvcFile { get; } public TimeSpan OpenTimeout { get; set; } public TimeSpan SendTimeout { get; set; } public TimeSpan ReceiveTimeout { get; set; } public TimeSpan CloseTimeout { get; set; } public uint MaximumExecutionTime { get; set; } public bool FailBalancerOnTimeout { get; set; } public uint MaximumNumberOfAttemptBeforeFailing { get; set; } #endregion #region Helper Methods private static string GetEndpointName(Uri address) { ArgumentValidator.IsNotNull(address, "address"); string endpointName = null; if (address.Scheme == Uri.UriSchemeHttp) endpointName = "http"; else if (address.Scheme == Uri.UriSchemeHttps) endpointName = "https"; else if (address.Scheme == Uri.UriSchemeNetTcp) endpointName = "tcp"; if (endpointName.IsNullOrEmpty()) throw new InvalidOperationException("Invalid address scheme " + address.Scheme); return endpointName; } #endregion } }
IUtilityService, UtilityService
A sample WCF service implemented within the service application. The service application can host as many services as needed.
using System.Security.Permissions;
using System.ServiceModel;
using Microsoft.SharePoint.Security;
using MyCorp.SP.ServiceApplication.DataContracts.Utility;
using MyCorp.SP.ServiceApplication.ServiceModel;
namespace MyCorp.SP.ServiceApplication.ServiceContracts
{
[ServiceContract(Namespace = Constants.ServiceApplicationNamespace),
SharePointPermission(SecurityAction.InheritanceDemand, ObjectModel = true),
SharePointPermission(SecurityAction.LinkDemand, ObjectModel = true)]
public interface IUtilityService
{
[FaultContract(typeof (MCServiceFault)), OperationContract]
HelloResponse Hello(HelloRequest request);
}
}
[/csharp]
The implementation:
using MyCorp.SP.ServiceApplication.DataContracts.Utility; using MyCorp.SP.ServiceApplication.Security; using MyCorp.SP.ServiceApplication.ServiceContracts; namespace MyCorp.SP.ServiceApplication.Services { public class UtilityService : BaseService, IUtilityService { #region IUtilityService Members public HelloResponse Hello(HelloRequest request) { ValidateAccess(); if (request == null || request.Message.IsNullOrEmpty()) { Log.Error(LogCategory.ServiceApplication, "An invalid request was made to the Hello service method by user " + base.UserLogin); throw new MCServiceException("Invalid Request"); } return new HelloResponse {Message = string.Concat("Hello ", request.Message)}; } #endregion private void ValidateAccess() { base.DemandAccess(MCServiceApplicationRights.UseUtilities); } } }
IUtilityServiceClient (in the core project), UtilityServiceClient, IServiceClientFactory (in the core project), MCServiceClientFactory
These are sample implementations of the MCServiceClient. The Utility service is 1 service within the service application.
namespace MyCorp.SP.ServiceClients { public interface IUtilityServiceClient { string Hello(string message); } }
The implementation:
using MyCorp.SP.ServiceApplication.DataContracts.Utility; using MyCorp.SP.ServiceApplication.ServiceContracts; using MyCorp.SP.ServiceClients; namespace MyCorp.SP.ServiceApplication.ServiceClients { public class UtilityServiceClient : MCServiceClient, IUtilityServiceClient { public string Hello(string message) { HelloResponse response = null; base.ExecuteOnChannel(c => response = c.Hello(new HelloRequest { Message = message }), true); return response.Message; } public override string EndpointSvcFile { get { return "util.svc"; } } } }
Accessed using a client service factory:
namespace MyCorp.SP.ServiceClients { public interface IServiceClientFactory { IUtilityServiceClient GetUtilityServiceClient(); } }
The implementation:
using MyCorp.SP.ServiceClients; namespace MyCorp.SP.ServiceApplication.ServiceClients { public class MCServiceClientFactory : IServiceClientFactory { #region IServiceClientFactory Members public IUtilityServiceClient GetUtilityServiceClient() { return new UtilityServiceClient(); } #endregion } }
Event Receiver Installer Code
This was a lot of code, and we’re not done. We still need to create some UI and show how client code will connect and use the services exposed as part of the service application. But I’ll do that in the next post.
But for now, I created some code as part of the Service Application Server project, in the Feature Event Receiver, to show how to provision and unprovision the service application. This will do all the work to create the service, service instance (on the local dev server), service proxy, service application and service application proxy, IIS application pool, database, and WCF service endpoints in IIS.
public override void FeatureActivated(SPFeatureReceiverProperties properties) { Log.InfoFormat(LogCategory.ServiceApplication, "{0:T} feature activated", typeof(MyCorpServiceApplicationInfrastructureEventReceiver)); var web = properties.Feature.Parent as SPWeb; #if DEBUG // NOTE: After a Library Service has been created for the first time, SharePoint’s service // application management will become aware of the new service application and allow // administrators to create or manage instances of the Service Application from // Central Administration. If more than one entry of 'Library Services' is appearing in // the New popup menu, than different service instance are created and can be removed by // calling the SPFarm.Local.Services.Remove method. Log.Info(LogCategory.ServiceApplication, "Creating service with service instance"); ServiceInstaller.CreateServiceWithServiceInstance(); //return; Log.Info(LogCategory.ServiceApplication, "Creating service applicaiton with service application proxy"); ServiceInstaller.CreateServiceApplicationWithApplicationProxy(); #endif } public override void FeatureDeactivating(SPFeatureReceiverProperties properties) { Log.InfoFormat(LogCategory.ServiceApplication, "{0:T} feature deactivating", typeof(MyCorpServiceApplicationInfrastructureEventReceiver)); var web = properties.Feature.Parent as SPWeb; #if DEBUG try { ServiceInstaller.RemoveServiceAndAllDependentObjects(); } catch { } #endif }
And we can now see the service application deployed in our SharePoint installation.
First the service and service instance:
The service application and service application proxy:
IIS WCF service:
The database:
The job definitions:
Testing the Service Application
To validate that the service application is working, we’ll run some quick tests using a test page. The code looks something like this:
protected void OnBtnClick(object sender, EventArgs e) { var clientFactory = Locator.Resolve(); var client = clientFactory.GetUtilityServiceClient(); try { this.lblMessage.Text = client.Hello(this.txtWorld.Text); } catch (Exception ex) { this.lblMessage.Text = string.Concat("ERROR: ", ex.Message); } }
Because we haven’t yet given anyone access rights to the utility service, we should get an access denied, which we do.
Let’s see if the Admin override works, by giving my user the custom Manage Utilities access right. Then we will verify that the custom Use Utilities access right works
And it works!
Now let’s force an error, which happens when we send a blank message:
Exactly the error we were expecting!
In the next post, I’ll finish the admin property pages for the service application and service application proxy, add some SharePoint CmdLets to manage this custom service app, and in my final post I will integrate a 3rd party service into this service application and show how one could leverage the service app within the SharePoint farm.
If you are reading this, you are AWESOME! You hung in there! THANKS!
Drop me a note/comment, feel free to take a look at the code. I’ll keep updating the same zip download as we move forward.
[button link=”https://skydrive.live.com/redir?resid=30FD0C7F694C1B3F!293&authkey=!AJ9mKTemY0vGR_4″ color=”primary” target=”_blank” size=”large” title=”Code Download” icon_before=”download”]Code Download[/button]
Have fun! Use at your own risk 🙂
Hi Matt,
Thanks a lot for sharing these great series of post on Custom Service Applications.
I also had to create a new custom service application and with all the basics implemented it was working fine for a while. After couple of redeployment and testing suddenly my application started failing. And i stated troubleshooting with a very basic hello world method that will return a string. But still it was failing.
The error I’m getting is “The remote server returned an error: (404) Not Found.” ” There was no endpoint listening at http://monkey:32843/e5326f085f454762871477e513fc1bee/MFTService.svc ”
This happens it executes the line “loadBalancer.BeginOperation();”
Return service url from the Load balancer is correct and i can verify it through browser.
I found another post explaining the same issue, but i have done all necessary as explained in this post.
http://sharepoint.stackexchange.com/questions/20345/there-is-no-default-service-endpoint-for-service-application-w-custom-service
Can you please let me know what may be wrong that I’m doing?
Thanks and regards,
Kamil
How do I reference the Microsoft.Sharepoint.Administration and Microsoft.Sharepoint.Security namespaces? Where do I get the DLL for those?
Thank you very mutch for the answer. You are right.. those are fundamental components.
I still have a lot to learn and I REALLY want to do this.
The next weeks i will dedicate my freetime to this – i hope it is ok if i get back to you from time to time.
No problem.
Hi Matt
I understood the basics. To deeply understand what is going on, I need to get my own work into it.
It is a very elegant and nice solution – but for me it is not a 101 because I can not find a starting point.
What is needed – what are nice extras?
The Problem with a not so basic-solution: I need to learn every bit of the code – after that i know if I need it or not. Massive overhead for a starter like me.
What i hope to learn:
I have ServiceStack, XSockets, Node.js(on IIS) and WCF Services. Some Services use SQL, some use Redis, other use MongoDB – they are all working & tested.
It would be the so GREAT to use them in a SharePoint infrastructure – for me this is the marriage between work and hobby.
Maybee i know to little, but what do you want the reader to know in a 101 ? 🙂
In my opinion, you need to start a bit more basic and build on that.
When you say that this is the most basic solution – then i have to learn a lot more before i can begin this 101 🙂
Hey Thomas,
Thanks for this great feedback! Here are some thoughts in response.
My goal in this series was to provide a basic, but comprehensive look at developing a service application with the essential components needed to cover most scenarios a development team might need. I state this is a 101 because it’s a basic overview of the fundamental components and how a base solution would be structured to accomplish the task. That said, building service applications is not a 101 task in itself, and that’s why in my first post I present this mostly as a viable architecture for ISVs and mid to large companies who can swing the development expense. That said, I think it is perfectly appropriate for any SharePoint developer to take on as it really immerses you into one fundamental aspect of SharePoint architecture and thereby gives you a better understanding of how all the SharePoint service apps work. Doing this as a hobby project is how I built my first service app 3 years ago, and even though that was 3 years ago, my experience is that very few developers have attempted to do this still at this time. It pays off though, as one of my clients for example has developers new to working with SharePoint directly successfully adding features and functionality to their SharePoint farm by adding services using this architecture, and is focusing on UX design for front-end work to consume the services.
There are some other 101 examples of service applications posted on the web that can help as well (like this calculator service, this sridharserviceapplication, also here, the Wingtip service, and the autocompletedemo).
Regarding your requirements, it looks like you have some great technologies to work with at your disposal. They all are good candidates for inclusion into a service app in some way or another, imo. I think the starting point is to figure out how you want things to be integrated, and this will vary depending on what you are integrating, and how these 3rd party systems function within your eco-system. Personally, I use ServiceStack as an API layer for developers to use in building SharePoint UX components (web parts, javascript apis, etc…). These APIs then use service application client classes (I’ll go over these in my next post in more detail) to call service application services using WCF. This minimizes the database interaction on the WFEs, and pushes that interaction to the app servers and also allows for a clean security paradigm managed within central admin. As far as XSockets and Node.js goes, since these are browser interactions back to services hosted on a server, it seems to me that the service application could be the host of the “connection information” here (i.e.: endpoint information) and you would add this endpoint information as properties to the Service Application with the Persist attribute and make those editable in the manageapp.aspx file. I did a quick test and I was able to put XSockets.NET into SharePoint by copying the javascript and html from the demo app into a layout page, so this works nice. (Thanks for the tip on that project 🙂 ). As far as external SQL, NoSQL, and Cache storages, you could manage the connection string properties on the service application class, on the service app proxy, in the service app database, or as farm properties, it’s really up to you. If you want your WFE to be able to connect to MongoDB directly, you could store the property on the service app proxy for example and avoid a WCF round trip, if you want to wrap some security around your calls and control things at the app server level, manage the connections within your service app and exchange DTOs back and forth between the WFEs and app server(s).
While you won’t build a service app overnight, simply ’cause it takes time, thought, optimization, I think it pays off.
[…] SharePoint 2010 Service Application Development 101 – Base Solution (Matt Cowan) […]