Mixing Mobile Friendly Forms based auth with NTLM Windows auth in SharePoint

One of the complaints I hear often is when users on mobile devices, mainly iOS, attempt to login to SharePoint.  Typically they are presented with a popup box asking for their username and password but there isn’t an option to save the password.  This is because our SharePoint web applications are using NTLM windows authentication because this is the most compatible with desktop PCs and provides the most functionality in SharePoint.  Apple is decidedly “nerfing” their safari browser to force more apps to be sold which makes them more money.  Part of this is them not providing an option to save passwords for NTLM authenticated sites and thus requiring apps to be made to support browsing SharePoint.

I’ve been working on this problem for a while.  Because of BI and some other features we couldn’t move away from Windows authentication for SharePoint.  So last year a coworker and I worked on creating our own browser app in iOS which would save user’s credentials and allow access to our SharePoint sites.  The problem with this approach is now we are developing a commodity, a web browser, which could end up taking a lot of time and effort for not much benefit.  After a long year last year working on many projects I got some breathing room at the beginning of this year and my brain decided to work on this problem and I came up with an idea.

Since I know SharePoint takes a windows principal and converts it into a windows claim, what if I could inject myself right BEFORE it converts the principal into a claim and replace the principal with a user of my choice.  I reflectored the code in Microsoft.SharePoint.IdentityModel.SPWindowsClaimsAuthenticationHttpModule and found the right spot and after building my own httpmodule and configuring it to run before SharePoint’s (see web.config example below), I was able to change the currently logged in user to be one of my choice.  This was great because this meant that I could potentially create a forms based experience for mobile devices and then tell SharePoint to use that user instead of thinking we weren’t authenticated.  The reason forms based authentication is needed is because iOS supports remembering passwords for forms based authentication.

SharePoint web.config

<system.webServer>
...
 <modules runAllManagedModulesForAllRequests="true">
 ...
 <add name="WindowsFormLogin" preCondition="integratedMode" type="SharePoint.HttpModules.WindowsFormLogin, SharePoint.HttpModules, Version=1.0.0.0, Culture=neutral, PublicKeyToken=***********" />
 <add name="SPWindowsClaimsAuthentication" type="Microsoft.SharePoint.IdentityModel.SPWindowsClaimsAuthenticationHttpModule, Microsoft.SharePoint.IdentityModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
 ...
 </modules>
...
</system.webServer>

Notice in the httpmodule below that I am checking to see if the useragent of the device is a mobile device on line 21.  Then I am checking if the status being returned is a 401 (unauthorized) on line 39.  Originally those were the only rules I had in place before I redirected to my forms login page.  But then I ran into some issues with the SharePlus app, which uses the same useragent as safari, trying to use webservices and requesting files and it didn’t understand our new authentication.  So i placed some other checks in there to determine whether it was a request from a browser or from an app.  Apps never have referrers set, so I check that first (line 59), and then I have a few other checks in place including looking at the querystring for web=1 (line 69) which is what SP uses to open files in the office web apps (office online).  So if i think this request is coming from a mobile device and it’s not an app, then I redirect to my special login page (line 85), otherwise I add some headers (line 90) which may not be needed but potentially apps such as office might be interested in.

HttpModule – PreSendRequestHeaders

public class WindowsFormLogin : IHttpModule
{
	public void Init(HttpApplication context)
	{
		if (context == null)
		{
			return;
		}
		context.AuthenticateRequest += context_AuthenticateRequest;
		context.PreSendRequestHeaders += context_PreSendRequestHeaders;
	}

	void context_PreSendRequestHeaders(object sender, EventArgs e)
	{
		HttpApplication httpApplication = sender as HttpApplication;
		HttpContext context = httpApplication.Context;

		if (context == null) return;
		if (context.Request == null) return;
		if (context.Request.Browser == null) return;
		if (false == context.Request.Browser.IsMobileDevice) return;
		

		if (context.Request.Cookies[Constants.MOBILE_LOGIN_EXCLUDE_COOKIE_NAME] != null)
		{
			string cookieValue = context.Request.Cookies[Constants.MOBILE_LOGIN_EXCLUDE_COOKIE_NAME].Value;
			bool exclude;
			if (false == bool.TryParse(cookieValue, out exclude))
			{
				exclude = false;
			}

			if (exclude) return;
		}

		string userAgent = context.Request.ServerVariables["HTTP_USER_AGENT"];
		if (userAgent.Contains("AppleWebKit") &&
		    false == userAgent.Contains("Safari")) return;

		int status = context.Response.StatusCode;
		switch (status)
		{
			case 401:
				if (IsPageRequest(context))
		                    {
		                        HandlePageRequest(context);
		                    }
		                    else
		                    {
		                        HandleNonPageRequest(context);
		                    }
				
				break;
		}
	}

private bool IsPageRequest(HttpContext context)
        {
            if (context.Request.Url == null) return false;
            if (string.IsNullOrEmpty(context.Request.Url.AbsolutePath)) return false;

            if (context.Request.UrlReferrer != null &&
                false == string.IsNullOrEmpty(context.Request.UrlReferrer.ToString()))
            {
                return true;
            }

            if (context.Request.Url.AbsolutePath.ToLower().EndsWith(".aspx")) return true;
            if (context.Request.Url.AbsolutePath.Contains("/_layouts/")) return true;
            if (context.Request.Url.AbsolutePath.Contains("/_")) return false;

            if (context.Request.QueryString["web"] != null &&
                context.Request.QueryString["web"] == "1")
            {
                return true;
            }

            try
            {
                if (SPContext.Current.Web.ServerRelativeUrl.ToLower() == context.Request.Url.AbsolutePath.ToLower()) return true;
            }
            catch { }

            return false;
        }

        private void HandlePageRequest(HttpContext context)
        {
            context.Response.Redirect("/_mobilelogin/Home/Login?ReturnUrl=" + HttpUtility.UrlEncode(context.Request.Url.ToString()));
        }

        private void HandleNonPageRequest(HttpContext context)
        {
            if (context.Response.Headers["X-Forms_Based_Auth_Required"] == null)
            {
                context.Response.Headers.Add("X-Forms_Based_Auth_Required", "/_mobilelogin/Home/Login?ReturnUrl=" + HttpUtility.UrlEncode(context.Request.Url.ToString()));
            }
            if (context.Response.Headers["X-Forms_Based_Auth_Return_Url"] == null)
            {
                Uri relativeUri;
                Uri.TryCreate("/" + SPUtility.ContextLayoutsFolder + "/error.aspx", UriKind.Relative, out relativeUri);
                string absoluteUri2 = new Uri(SPAlternateUrl.ContextUri, relativeUri).AbsoluteUri;
                context.Response.Headers.Add("X-Forms_Based_Auth_Return_Url", absoluteUri2);
            }

            if (context.Response.Headers["X-MSDAVEXT_Error"] == null)
            {
                string value = "917656; " + HttpUtility.HtmlAttributeEncode(SPResource.GetString("FormsAuthenticationNotBrowser", new object[0]));
                context.Response.Headers.Add("X-MSDAVEXT_Error", value);
            }
        }


}

I decided to create a ASP.NET MVC web application for my login page.  For things to work as expected, this needs to be hosted in your SharePoint farm.  I decided to add it as a virtual directory called _mobilelogin in IIS under my SharePoint web applications.  Because I need a windows principal object later, I need to store the person’s username and password somewhere I can get to it later.  There are a few different ways of handling this, each with their own pros and cons but to keep things simple for this blog post, I will use the FormsAuthenticationTicket (line 70) that normal ASP.NET Forms based authentication uses to store both the userid and password.  This ticket gets encrypted (line 78) and stored as a cookie in the browser of the mobile device (line 84).  Since the password is sensitive, it will be doubly encrypted because it gets encrypted once (line 75) and then put into the ticket which gets encrypted again.

MVC Model and Controller

public class LoginModel
{
	public string UserName { get; set; }
	public string Password { get; set; }
	public bool CookieSet { get; set; }
	public bool RememberMe { get; set; }
}
	
public class HomeController : Controller
{
	[HttpGet]
	[OutputCache(NoStore = true, Duration = 0, VaryByParam = "None")] 
	public ActionResult Login()
	{
		LoginModel mdl = new LoginModel();
		mdl.RememberMe = true;
		try
		{
			if (Request.Cookies[Constants.MOBILE_LOGIN_COOKIE_NAME] != null)
			{

				string cookieValue = Request.Cookies[Constants.MOBILE_LOGIN_COOKIE_NAME].Value;

				if (false == string.IsNullOrEmpty(cookieValue))
				{
					FormsAuthenticationTicket ticket = FormsAuthentication.Decrypt(cookieValue);
					if (ticket.Expired)
					{
						Utility.LogoutHelper(Response);
					}
					else
					{
						mdl.UserName = ticket.Name;
						mdl.CookieSet = true;
					}
				}
			}
		}
		catch (Exception e)
		{
			ViewBag.ErrorMsg = e.Message;
		}

		return View(mdl);
	}

	[HttpPost]
	[OutputCache(NoStore = true, Duration = 0, VaryByParam = "None")] 
	public ActionResult Login(LoginModel mdl)
	{
		try
		{
			if (false == string.IsNullOrEmpty(mdl.UserName) &&
				false == string.IsNullOrEmpty(mdl.Password))
			{
				string username = mdl.UserName;
				if (false == Utility.IsAnEmailAddress(username) &&
					string.IsNullOrEmpty(Utility.ExtractDomain(username)))
				{
					username = ConfigurationManager.AppSettings["DefaultDomain"] + "\\" + username;
				}
				WindowsIdentity identity = Utility.Logon(username, mdl.Password);

				DateTime expiration = DateTime.Now.AddMinutes(Constants.MOBILE_LOGIN_TTL_SHORT_MIN);
				if (mdl.RememberMe)
				{
					expiration = DateTime.Now.AddDays(Constants.MOBILE_LOGIN_TTL_LONG_DAYS);
				}

				FormsAuthenticationTicket ticket = new FormsAuthenticationTicket(1,
					username,
					DateTime.Now,
					expiration,
					mdl.RememberMe,
					Utility.Protect(mdl.Password, Constants.MOBILE_LOGIN_SALT),
					FormsAuthentication.FormsCookiePath);

				HttpCookie cookie = new HttpCookie(Constants.MOBILE_LOGIN_COOKIE_NAME, FormsAuthentication.Encrypt(ticket));
				if (mdl.RememberMe)
				{
					cookie.Expires = ticket.Expiration;
				}
				Utility.SetAttributes(cookie);
				Response.Cookies.Add(cookie);

				HttpCookie cookieExclude = new HttpCookie(Constants.MOBILE_LOGIN_EXCLUDE_COOKIE_NAME, false.ToString());
				Utility.SetAttributes(cookieExclude);
				cookieExclude.Expires = DateTime.Now.AddDays(-1);
				Response.Cookies.Add(cookieExclude);
				
				Redirect();
			}
			else
			{
				ViewBag.ErrorMsg = "Please fill in both the UserName and Password fields.";
			}
		}
		catch (Exception e)
		{
			ViewBag.ErrorMsg = e.Message;
		}

		return View(mdl);
	}

	public ActionResult Logout()
	{
		Utility.LogoutHelper(Response);

		return View();
	}

	[OutputCache(NoStore = true, Duration = 0, VaryByParam = "None")] 
	public ActionResult Exclude()
	{
		HttpCookie cookieExclude = new HttpCookie(Constants.MOBILE_LOGIN_EXCLUDE_COOKIE_NAME, true.ToString());
		Utility.SetAttributes(cookieExclude);
		Response.Cookies.Add(cookieExclude);

		Utility.LogoutHelper(Response);
		Redirect();

		return View();
	}

	private void Redirect()
	{
		string returnUrl = "/";
		if (false == string.IsNullOrEmpty(Request.QueryString["ReturnUrl"]))
		{
			returnUrl = HttpUtility.UrlDecode(Request.QueryString["ReturnUrl"]);
		}

		Response.Redirect(returnUrl);
	}
}

MVC View

@model LoginModel

@{
    ViewBag.Title = "Login";
}


<div class="container">
    <div class="row">
        <div class="col-sm-12">
            @using (Html.BeginForm())
            {
                <div class="form-horizontal" style="margin-top:90px;width:90%">
                    <div class="form-group">
                        <div class="col-md-offset-3 col-sm-9">
                            <h4>This is a special login page just for mobile devices.  If you feel you reached this page in error, @Html.ActionLink("click here to restore normal login functionality", "Exclude", "Home").</h4>
                        </div>
                    </div>
                    <div class="form-group">
                        <div class="col-md-offset-3 col-sm-9">
                            @if (false == string.IsNullOrEmpty(ViewBag.ErrorMsg))
                            {
                                <p class="error">@ViewBag.ErrorMsg</p>
                            }
                        </div>
                    </div>
                    @if (Model.CookieSet)
                    {
                        <div class="form-group">
                            <div class="col-md-offset-3 col-sm-9">
                                <h4>
                                    You are currently logged in as: @Model.UserName <br />
                                    @Html.ActionLink("Click here to logout", "Logout", "Home", new { ReturnUrl = Request.QueryString["ReturnUrl"] }, new { } ).
                                </h4>
                            </div>
                        </div>
                        <div class="form-group">
                            <div class="col-md-offset-3 col-sm-9">
                                <h4>
                                    @if (string.IsNullOrEmpty(Request.QueryString["ReturnUrl"]))
                                    {
                                        <a href="/">Return to site</a>
                                    }else{
                                        <a href="@HttpUtility.UrlDecode(Request.QueryString["ReturnUrl"])">Return to site</a>
                                    }
                                </h4>
                            </div>
                        </div>
                    }
                    else
                    {
                    <div class="form-group">
                        @Html.Label("User Name", new { @class = "col-sm-3 control-label" })
                        <div class="col-md-9">
                            @Html.TextBoxFor(m => m.UserName, new { @class = "form-control" })
                            @Html.ValidationMessageFor(m => m.UserName, null, new { @class = "help-inline" })
                        </div>
                    </div>
                    <div class="form-group">
                        @Html.LabelFor(m => m.Password, new { @class = "col-sm-3 control-label" })
                        <div class="col-md-9">
                            @Html.PasswordFor(m => m.Password, new { @class = "form-control" })
                            @Html.ValidationMessageFor(m => m.Password, null, new { @class = "help-inline" })
                        </div>
                    </div>
                    <div class="form-group">
                        <div class="col-md-offset-3 col-sm-9">
                            <input type="submit" value="Log in" class="btn btn-default" />
                            @Html.Label("Remember Me")
                            @Html.CheckBoxFor(m => m.RememberMe, new { @checked = "checked" })
                        </div>
                    </div>
                    <div class="form-group">
                        <div class="col-md-offset-3 col-sm-9">
                            <h4>If you have previously saved your password in your browser, select the username field above and then select autofill password at the top of the onscreen keyboard.</h4>
                        </div>
                    </div>
                    }
                    <br />
                </div>
            }
        </div>
    </div>
</div>

After we set the cookie in our MVC application, we need to redirect back to where we came from.  Then the HttpModule from earlier kicks in but on a different event: AuthenticateRequest.  Notice on line 26 where our Ticket gets unencrypted and I check to make sure it is still valid and not expired.  After that I get a WindowsIdentity object by logging in as the user with their username and password (line 33).  I then create a WindowsPrincipal object from the identity and assign it to the user property of the HttpContext (line 36).  This HttpContext.User property is what the SPWindowsClaimsAuthentication HttpModule will look at when creating the Windows Claim which SharePoint then uses everywhere.  I also hijack the signout page on line 38 to redirect to our forms logout page when we are logged in using our forms method.

HttpModule – AuthenticateRequest

public class WindowsFormLogin : IHttpModule
{
	public void Init(HttpApplication context)
	{
		if (context == null)
		{
			return;
		}
		context.AuthenticateRequest += context_AuthenticateRequest;
		context.PreSendRequestHeaders += context_PreSendRequestHeaders;
	}

	void context_AuthenticateRequest(object sender, EventArgs e)
	{
		HttpApplication httpApplication = sender as HttpApplication;
		HttpContext context = httpApplication.Context;

		try
		{
			if (context.Request.Cookies[Constants.MOBILE_LOGIN_COOKIE_NAME] == null) return;

			string cookieValue = context.Request.Cookies[Constants.MOBILE_LOGIN_COOKIE_NAME].Value;

			if (false == string.IsNullOrEmpty(cookieValue))
			{
				FormsAuthenticationTicket ticket = FormsAuthentication.Decrypt(cookieValue);
				if (ticket.Expired)
				{
					throw new ApplicationException("Mobile Login Expired");
				}

				string user = ticket.Name;
				WindowsIdentity identity = Utility.Logon(user, Utility.Unprotect(ticket.UserData, Constants.MOBILE_LOGIN_SALT));

				context.Request.ServerVariables["LOGON_USER"] = user;
				context.User = new WindowsPrincipal(identity);

				int index = context.Request.RawUrl.ToLower().IndexOf("/_layouts/15/signout.aspx");
				if (index >= 0)
				{
					string returnUrl = "/";
					if (index > 0)
					{
						returnUrl = context.Request.RawUrl.ToLower().Substring(0, index);
					}
					if (context.Request.UrlReferrer != null &&
						false == string.IsNullOrEmpty(context.Request.UrlReferrer.ToString()))
					{
						returnUrl = context.Request.UrlReferrer.ToString();
					}
					context.Response.Redirect("/_mobilelogin/Home/Logout?ReturnUrl=" + HttpUtility.UrlEncode(returnUrl));
				}
			}
		}
		catch (Exception ex)
		{
			//MobileLogin Login Error
			Utility.LogoutHelper(context);
		}
	}
}

I have used a few helper methods in my code above so below is this code which should enable you to put this solution together in your own environment.  Also, the MVC login app pool service account and your SharePoint app pool service account need special permissions on the server in order for them to login as another user on line 61.  This permission is called “Act as part of the operating system” which can be found in user rights assignment in local security policies.

Utility Class

public class Constants{
	public const string MOBILE_LOGIN_COOKIE_NAME = "MobileLogin";
	public const string MOBILE_LOGIN_EXCLUDE_COOKIE_NAME = "MobileLoginExcluded";
	public const int MOBILE_LOGIN_TTL_LONG_DAYS = 90;
	public const int MOBILE_LOGIN_TTL_SHORT_MIN = 90;
	public const string MOBILE_LOGIN_SALT = "TEST 123";
}

public class Utility
{
	public static string Protect(string text, params string[] purpose)
	{
		if (string.IsNullOrEmpty(text))
			return null;

		byte[] stream = Encoding.UTF8.GetBytes(text);
		byte[] encodedValue = MachineKey.Protect(stream, purpose);
		return HttpServerUtility.UrlTokenEncode(encodedValue);
	}
	public static string Unprotect(string text, params string[] purpose)
	{
		if (string.IsNullOrEmpty(text))
			return null;

		byte[] stream = HttpServerUtility.UrlTokenDecode(text);
		byte[] decodedValue = MachineKey.Unprotect(stream, purpose);
		return Encoding.UTF8.GetString(decodedValue);
	}

	public static WindowsIdentity Logon(string username, string password)
	{
		string domain = Utility.ExtractDomain(username);
		username = Utility.ExtractName(username);

		IntPtr handle = new IntPtr(0);
		handle = IntPtr.Zero;

		const int LOGON32_LOGON_NETWORK = 3;
		const int LOGON32_PROVIDER_DEFAULT = 0;

		// attempt to authenticate domain user account
		bool logonSucceeded = LogonUser(username, domain, password, LOGON32_LOGON_NETWORK, LOGON32_PROVIDER_DEFAULT, ref handle);

		if (!logonSucceeded)
		{
			// if the logon failed, get the error code and throw an exception
			int errorCode = Marshal.GetLastWin32Error();
			throw new Exception("User logon failed. Error Number: " + errorCode);
		}

		// if logon succeeds, create a WindowsIdentity instance
		WindowsIdentity winIdentity = new WindowsIdentity(handle, "NTLM");

		// close the open handle to the authenticated account
		CloseHandle(handle);

		return winIdentity;
	}

	[DllImport("advapi32.dll", SetLastError = true)]
	private static extern bool LogonUser(string lpszUsername,
										string lpszDomain,
										string lpszPassword,
										int dwLogonType,
										int dwLogonProvider,
										ref IntPtr phToken);

	[DllImport("kernel32.dll", CharSet = CharSet.Auto)]
	private static extern bool CloseHandle(IntPtr handle);
	
	public static bool IsAnEmailAddress(string text)
	{
		if (text == null) return false;
		if (text.Length <= 0) return false;
		if (text.IndexOf(" ") >= 0) return false;

		string patternLenient = @"\w+([-+.]\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*";
		Regex reLenient = new Regex(patternLenient);
		string patternStrict = @"^(([^<>()[\]\\.,;:\s@\""]+"
			+ @"(\.[^<>()[\]\\.,;:\s@\""]+)*)|(\"".+\""))@"
			+ @"((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"
			+ @"\.[0-9]{1,3}\])|(([a-zA-Z\-0-9]+\.)+"
			+ @"[a-zA-Z]{2,}))$";
		Regex reStrict = new Regex(patternStrict);
		return reStrict.IsMatch(text);
	}

	public static string ExtractDomain(string UserName)
	{
		if (UserName.IndexOf(Convert.ToChar("\\")) > 0)
		{
			string[] fullDomainUserName = UserName.Split(new char[] { '\\' }, 2);
			return fullDomainUserName[0];
		}
		else
		{
			return string.Empty;
		}
	}

	public static string ExtractName(string name)
	{
		if (name.IndexOf(Convert.ToChar("\\")) > 0)
		{
			string[] fullDomainName = name.Split(new char[] { '\\' }, 2);
			return fullDomainName[fullDomainName.Length - 1];
		}
		else
		{
			return name;
		}
	}
	
	public static void SetAttributes(HttpCookie cookie, HttpContext context)
	{
		cookie.HttpOnly = true;
		string domain = GetRootCookieDomain(GetBasePath(context));
		if (false == string.IsNullOrEmpty(domain) &&
			domain.Contains('.'))
		{
			cookie.Domain = domain;
		}
	}

	public static void LogoutHelper(HttpContext context)
	{
		try
		{
			HttpCookie cookie = new HttpCookie(Constants.MOBILE_LOGIN_COOKIE_NAME, string.Empty);
			cookie.Expires = DateTime.Now.AddDays(-1);
			SetAttributes(cookie, context);

			context.Response.Cookies.Add(cookie);
		}
		catch (Exception err)
		{
			//MobileLogin Logout Cookie Error
		}
	}
	
	public static string GetRootCookieDomain(string url)
	{
		url = GetBasePathFromUrl(url);
		if (string.IsNullOrEmpty(url)) return url;
		Uri uri = new Uri(url);
		string host = uri.Host;
		if (false == host.Contains('.')) return uri.Host;

		string[] hostParts = host.Split('.');
		string domain = "." + hostParts[hostParts.Length - 2] + "." + hostParts[hostParts.Length - 1];

		return domain;
	}
	
	public static string GetBasePathFromUrl(string url)
	{
		string newUrl = string.Empty;
		if (url.Contains("://"))
		{
			newUrl = url.Substring(0, url.IndexOf("://") + 3);
			url = url.Substring(url.IndexOf("://") + 3, url.Length - url.IndexOf("://") - 3);
			if (url.Contains("/"))
			{
				newUrl += url.Substring(0, url.IndexOf("/"));
			}
			else
			{
				newUrl += url;
			}
		}
		return newUrl;
	}
	
	public static string GetBasePath(HttpContext context)
	{
		string basepath = "";
		if (context.Request.ServerVariables["HTTPS"].ToString().ToUpper().Equals("ON"))
		{
			basepath = "https://";
		}
		else
		{
			basepath = "http://";
		}
		basepath += context.Request.ServerVariables["SERVER_NAME"].ToString();
		if (!context.Request.ServerVariables["SERVER_PORT"].ToString().Equals("80") &&
			!context.Request.ServerVariables["SERVER_PORT"].ToString().Equals("443"))
		{
			basepath += ":" + context.Request.ServerVariables["SERVER_PORT"].ToString();
		}
		return basepath;
	}
}

Edit 05/18/2016:

Added a useragent check before the 401 status check to help with apps like the SharePoint iOS app and SharePlus which already know how to connect to SharePoint using NTLM.

Unable to Manually Start Workflows

Recently, after we finished migrating our large external farm from SharePoint 2010 to SharePoint 2013, I started hearing complaints that some user’s could not manually start workflows.  They would either get a 403 Forbidden or get the “Sorry, you don’t have access to this page” message.

2014-09-16_8-57-02

The stack trace on the back end produced the following:

Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(HttpContext context) at Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(Exception ex) at Microsoft.SharePoint.SPSecurableObject.CheckPermissions(SPBasePermissions permissionMask) at Microsoft.SharePoint.WorkflowServices.StoreSubscriptionService.EnumerateSubscriptionsByEventSource(Guid eventSourceId) at Microsoft.SharePoint.WorkflowServices.ApplicationPages.WorkflowPage.ConstructStartArray() at Microsoft.SharePoint.WorkflowServices.ApplicationPages.WorkflowPage.OnLoad(EventArgs e)

Figuring out the problem just took a little investigation.  I recently used this as a Demo for a SharePoint session I did recently at my local SharePoint Saturday.  From that session I have a video of the issue and how I determined the cause.  Skip to about 2:15 for the start of the issue and investigation.  Click here to see the video

So basically the new SharePoint 2013 Workflow Manager is checking for Contribute access at the Site (SPWeb) scope when determining if someone should see the list of workflows for a list (see first line of the method in the code below).

// Microsoft.SharePoint.WorkflowServices.StoreSubscriptionService
public override WorkflowSubscriptionCollection EnumerateSubscriptionsByEventSource(Guid eventSourceId)
{
	this.context.Web.CheckPermissions(SPBasePermissions.EditListItems);
	WorkflowStore workflowStore = new WorkflowStore(this.context.Web);
	eventSourceId = StoreSubscriptionService.ConvertToGuidToken(eventSourceId, this.context.Web);
	WorkflowFile[] files = workflowStore.QueryWithGuid("0x0100AA27A923036E459D9EF0D18BBD0B9587", StoragePublishState.Unchanged, "WSEventSourceGUID", eventSourceId);
	return this.ConvertToWorkflowSubscriptionCollection(files);
}

So in our scenario, our users had read access to the site but contribute access on the list.  This should allow them to start the workflow and it did in SP 2010.  I currently have a Design Change Request (DCR) open with MS regarding this issue so hopefully it will be fixed soon.  Once I hear more, I will update this post.  In the meantime there seems to be two workarounds.

Workaround 1:  Earlier in the code I determined that if you DON’T setup and connect a SharePoint 2013 Workflow Manager to the farm, then this code will never run and thus it will work just like it did in SP 2010.  Of course the issue with this workaround is you can’t have any SP 2013 workflows.

Workaround 2:  Basically give everyone that needs to manually start workflows contribute access to the site, and then break inheritance to all lists and libraries where the user DOESN’T need contribute access and remove their contribute access.

We ended up implementing the second workaround above which sucks for my users.  I am hoping this is fixed soon as I have also seen other people complaining about it (http://sharepoint.stackexchange.com/questions/115311/manually-start-sharepoint-2010-workflow-in-sharepoint-2013-farm/)

Getting Active Directory UserId from Windows Claim in SharePoint 2013

We’ve always used NTLM for our SharePoint authentication but in SharePoint 2013, claims is the preferred authentication method.  Fortunately, SharePoint 2013 ships with something called Windows Claims.  This seems to work the same as the NTLM auth from before but that windows auth is converted into a claim that SharePoint can use.

This change means that your userid would look something like this:

i:0#.w|contoso\chris

instead of this:

contoso\chris

Sometimes when calling other services, you need the windows userid and not the claim userid.  So for these instances, I’ve created a few helper methods.

//Regex needs more testing
public const string CLAIMS_REGEX = @"(?<IdentityClaim>[ic])?:?0(?<ClaimType>.)(?<ClaimValueType>.)(?<AuthMode>[wstmrfc])(\|(?<OriginalIssuer>[^\|]*))?(\|(?<ClaimValue>.*))";
 
public static string GetAdUserIdForClaim(string login)
{
    string userName = login;
 
    foreach (Match m in Regex.Matches(login, CLAIMS_REGEX, RegexOptions.IgnoreCase))
	{
		try
		{
			if (m.Groups["AuthMode"].Captures[0].Value.ToLower() == "w")
			{
				userName = m.Groups["ClaimValue"].Captures[0].Value;
			}
		}
		catch { }
	}
    return userName;
}
 
public static string GetClaimForAdUserId(string login)
{
    string userName = login;
    SPClaimProviderManager mgr = SPClaimProviderManager.Local;
    if (mgr == null) return userName;
 
    SPClaim claim = new SPClaim(SPClaimTypes.UserLogonName, login, "http://www.w3.org/2001/XMLSchema#string", SPOriginalIssuers.Format(SPOriginalIssuerType.Windows));
    userName = mgr.EncodeClaim(claim);
 
    return userName;
}
 
public static bool IsLoginClaims(string login)
{
    Regex re = new Regex(CLAIMS_REGEX, RegexOptions.IgnoreCase);
    return re.IsMatch(login);
}

First I made a regular expression to identify the different pieces of a claim (see http://social.technet.microsoft.com/wiki/contents/articles/13921.sharepoint-2013-and-sharepoint-2010-claims-encoding.aspx).  This allows me to effectively parse the claim for the windows login name (see GetAdUserIdForClaim).  This also allows me to validate whether a string is a claim or not (see IsLoginClaims).

Update 01-22-2015:

After some more usage, I found that I was being too limiting in the Claim Types and Claim Value Types in my regex.  I had based the options from the technet article above but I then ran into some other Claim Types when doing some work recently that were not in that article.  I then found this page:  http://blogs.msdn.com/b/scicoria/archive/2011/06/30/identity-claims-encoding-for-sharepoint.aspx which listed a lot more than the technet article.  It also now seems that almost any value could be there in the future.  Because of this I changed the regex in the code above to allow any value in those two fields.

Gotcha’s using Custom Web Parts and the Minimal Download Strategy

I’ve been playing around with some of my custom code in SharePoint 2013.  One of the issues I’ve been noticing is when I add any of my custom web parts to a page, the minimal download strategy (MDS) would failover to the normal asp.net webforms page.  You can tell the difference by looking at the url.  A url that looks like this:

/_layouts/15/start.aspx#/SitePages/Home.aspx

is using MDS.  Notice the actual page is start.aspx and then the page it is ajax loading (MDS) is /sitepages/home.aspx (the part after the hash(#)).  Whereas a normal asp.net webforms page url would look like this:

/SitePages/Home.aspx

They both look the same but when using MDS you get an added benefit of less being downloaded on each click and also smoother and snappier page changes.

Gotcha #1 – Decorate your assembly or class with MDSCompliant(true).  If your MDS isn’t working and you see this message in your ULS:

MDSLog: MDSFailover: A control was discovered which does not comply with MDS

then you will need to add the attribute (http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.webcontrols.mdscompliantattribute.aspx).  Here is an example of adding it to the class:

[MdsCompliant(true)]
public class MyWebPart : WebPart
{
	//...Code
}

And here is an example of adding it to an assembly (assemblyinfo.cs):

[assembly: MdsCompliant(true)]

Every control that is loaded on the page needs this attribute.  So if you are loading other controls in your web part, you’ll need to make sure they also have this attribute.

Gotcha #2 – Declarative user controls typically used for delegate controls need a codebehind and the attribute set as well.  I use a lot of delegate controls and then use features to swap out user controls in the delegate controls to add / remove functionality on the site.  Typically my user controls didn’t have a codebehind and would just add webparts, html or other controls in the markup.  The issue is if you want these controls to be able to be swapped out using MDS, then you will need to add a codebehind to the user control and decorate it with the MdsCompliant attribute.

So a normal user control like this:

<%@ Control Language="C#" AutoEventWireup="true"  %>
<%@ Register TagPrefix="MyWebParts" Namespace="MyWebParts"%>

<MyWebParts:WebPart runat="server" Title="WebPart"></MyWebParts:WebPart>

would need to be converted to this:

<%@ Assembly Name="$SharePoint.Project.AssemblyFullName$" %>
<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="MyControl.ascx.cs" Inherits="MyControl" %>
<%@ Register TagPrefix="MyWebParts" Namespace="MyWebParts"%>

<MyWebParts:WebPart runat="server" Title="WebPart"></MyWebParts:WebPart>

and with the following codebehind:

[MdsCompliant(true)]
public partial class MyControl : UserControl
{
    protected void Page_Load(object sender, EventArgs e)
    {
    }
}

I couldn’t figure out a way to decorate the usercontrol without using a code behind.  If anyone else knows how to do this, please comment or contact me with the info.  Thanks!

Gotcha #3 – Inline scripts are not allowed and will need to be added using the SPPageContentManager.  If you receive any of the following messages in your ULS logs you will need to look at your content.

MDSFailover: document.write
MDSFailover: document.writeln
MDSFailover: Unexpected tag in head
MDSFailover: script in markup

The first two are obvious; you can’t have any document.write or document.writeln’s in your html.  The third is a little less obvious.  According to a MS source, in the head these tags are not allowed:

  • Meta refresh tag
  • link tag for a stylesheet (text/css)
  • script tag
  • style tag
  • title tag
  • base tag

The fourth was a big kicker for me.  I had made the decision a few years ago to switch almost all of my webparts over to being XSLT rendered.  That means I had a lot of inline javascript and css in my xslt.  Luckily, I had previously also created a special webpart which could ajax load other webparts using an update panel and had solved the inline script issue before.  When I was writing my special ajax loading webpart before I found this page http://www.codeproject.com/Articles/21962/AJAX-UpdatePanel-With-Inline-Client-Scripts-Parser which showed how to extend the ootb update panel to automatically find inline scripts and register them to work correctly using ajax. I was able to slightly modify this code to work for my XSLT rendered webparts.

public bool IsAnAjaxDeltaRequest
{
	get
	{
		return false == String.IsNullOrEmpty(Context.Request.QueryString["AjaxDelta"]);
	}
}

protected override void Render(HtmlTextWriter output)
{
	base.Render(output);
	string html = GetHtml();
	if (IsAnAjaxDeltaRequest)
	{
		html = RegisterAndRemoveInlineClientScripts(this, this.GetType(), html);
	}
	output.Write(html);
}

public static readonly Regex REGEX_CLIENTSCRIPTS = new Regex(
"<script\\s((?<aname>[-\\w]+)=[\"'](?<avalue>.*?)[\"']\\s?)*\\s*>(?<script>.*?)</script>",
RegexOptions.Singleline | RegexOptions.IgnoreCase | RegexOptions.Compiled |
RegexOptions.ExplicitCapture);

public static string RegisterAndRemoveInlineClientScripts(Control control, Type type, string htmlsource)
{
	if (htmlsource.IndexOf("<script", StringComparison.CurrentCultureIgnoreCase) > -1)
	{
		MatchCollection matches = REGEX_CLIENTSCRIPTS.Matches(htmlsource);
		if (matches.Count > 0)
		{
			for (int i = 0; i < matches.Count; i++)
			{
				string script = matches[i].Groups["script"].Value;
				string scriptID = script.GetHashCode().ToString();
				string scriptSrc = "";

				CaptureCollection aname = matches[i].Groups["aname"].Captures;
				CaptureCollection avalue = matches[i].Groups["avalue"].Captures;
				for (int u = 0; u < aname.Count; u++)
				{
					if (aname[u].Value.IndexOf("src",
						StringComparison.CurrentCultureIgnoreCase) == 0)
					{
						scriptSrc = avalue[u].Value;
						break;
					}
				}

				if (scriptSrc.Length > 0)
				{
					SPPageContentManager.RegisterClientScriptInclude(control,
						type, scriptID, scriptSrc);
				}
				else
				{
					SPPageContentManager.RegisterClientScriptBlock(control, type,
						scriptID, script);
				}

				htmlsource = htmlsource.Replace(matches[i].Value, "");
			}

		}
	}
	return htmlsource;
}

Since this code will automatically register any script references it finds, make sure that the paths to your scripts are correct, otherwise MDS will silently failover without any ULS messages.

Update 5/3/2013:

I have found another potential MDS error message in ULS:

MDSLog: Master page version mismatch occurred in MDS

or in the ajax response if using fiddler:

versionMismatch

I was able to resolve this by browsing to another site and then back to the original site with the issue.  Weird one, if anyone knows more about this error, please contact me or comment below.

Process Drop Off Library Items Programmatically

I was working on a solution at work where they wanted to tag and approve all documents submitted to the site.  SharePoint 2010’s Content Organizer seemed like it would meet their needs.  I turned on approval for the drop off library and made the default content type be document.  That way when a external user submitted a document, they would just hit save and then the internal user who is approving it, would change it’s content type, fill out the metadata, and approve it.  Then the content organizer routing kicks in and puts it in the correct location.

This all worked great when working with a single document.  What didn’t work well is if you have several documents that needed to have the same metadata applied, approved, and routed.  I turned to Batch Edit on CodePlex for this piece.  I had used it before and with a few tweaks it worked great.  I ran into two issues though in this scenario.

  1. Batch Edit didn’t allow me to edit the content types, so we still had to go to each item individually to change it’s content type
  2. Batch Edit had no way to approve the items
  3. When saving an item programmatically in SharePoint, the routing rules did not process the item until the timer job ran (typically set to run at night once a day).

I wrote some code to fix #1 which was pretty straight forward and I can include the code here on the blog if someone leaves a comment saying they want it.  #2 involved adding a second button to the page that says save and approve and a little extra code on the save to check if that button was pressed and approve the item.  #3 was a lot harder than I thought it would be to get resolved.

Initially I was using OfficialFileCore.SubmitFile().  After realizing there were some additional permissions needed (Records Center Web Service Submitters SP group) I actually got it working.  It seems that OfficialFileCore.Submit() is mainly used to put a file INTO the drop off library and run the routing rules against that file.  The issue was that the files I needed to push through were already in the drop off library, and when using the OfficialFileCore.SubmitFile(), it made an extra copy of those files in the drop off library with a random name that I would somehow need to figure out and delete.  This is not exactly what I wanted and seemed overly complicated.

Remember that timerjob that runs nightly to process items in the drop off library and send emails out about items missing metadata, it’s called the Content Organizer Processing job  (Microsoft.Office.RecordsManagement.Internal.RecordsRepositoryJobDefinition).  I reflectored it and found it was calling an internal method named ProcessWaitingFiles which was looping through all of the files and calling another internal method named ProcessWaitingFile(SPListItem, bool).  This second one was really interesting to me since it only required the SPListItem needing to process and a bool which is whether or not moderation is enabled on the library.  Using reflection I was able to call this method and my files in the drop off library got routed just like I was doing them individually.

Here’s the code:

private void ProcessDropOffLibraryItem(SPWeb web, SPListItem item, bool listRequiresModeration)
{
	EcmDocumentRoutingWeb routingWeb = new EcmDocumentRoutingWeb(web);
	Type clsSPRequest = typeof(EcmDocumentRoutingWeb).Assembly.GetType("Microsoft.Office.RecordsManagement.RecordsRepository.EcmDocumentRoutingWeb", true);
	System.Reflection.MethodInfo processWaitingFile = clsSPRequest.GetMethod("ProcessWaitingFile", BindingFlags.Instance | BindingFlags.NonPublic);

	try
	{
		object result = processWaitingFile.Invoke(routingWeb, new object[] { item, listRequiresModeration });
	}
	catch (System.Reflection.TargetInvocationException ex)
	{
		string message = string.Format("Unable to route file {0}: {1}", item.File.ServerRelativeUrl, ex.Message);
		SPCriticalTraceCounter.AddDataToScope(67, "SP2010 BatchEdit", 1, message + ": " + ex.StackTrace);
	}
}