"Object ref not set to instance" from AuthServices/Acs with idp-initiated request

[dahlsailrunner]

Hi - I'm trying to get an IdP-initiated request to work with Okta and IdentityServer3, and overcame the error where "unsolicited requests" were not supported with the following code:

 authServicesOptions.IdentityProviders.Add(new IdentityProvider(
            new EntityId("http://www.okta.com/exk4yxtgy7ZzSDp8e0h7"), authServicesOptions.SPOptions)
        {
            LoadMetadata = true,
            MetadataUrl = new Uri("https://dev-490944.oktapreview.com/app/exk4yxtgy7ZzSDp8e0h7/sso/saml/metadata"),
            AllowUnsolicitedAuthnResponse = true

        });

Now I have okta set to send the request here to the AuthServices/Acs endpoint, but I get the following error:

[NullReferenceException: Object reference not set to an instance of an object.]
Kentor.AuthServices.Owin.<AuthenticateCoreAsync>d__1.MoveNext() +576
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +13908768
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +61
Kentor.AuthServices.Owin.<InvokeAsync>d__e.MoveNext() +464
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +13908768
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +61
Microsoft.Owin.Security.Infrastructure.<Invoke>d__0.MoveNext() +540

I've included an image below of some of the config within Okta -- I'm wondering if I need a different endpoint configured for the DestinationUrl or something (based on the Okta docs, it doesn't seem like this should be different)? Ultimately I want to get into my custom IdentityServer3 UserService so that I can do my claims transformation and redirect the user to the real destination page (the "home" page of my real site -- not within id server), but it gets stuck in the AuthServices endpoint. Can you give me any guidance that might help move me along? Maybe I need some additional code in a callback somewhere?

By the way, I'll be happy to update the documentation with anything that I discover to get this working. :)

Thanks in advance -
Erik

[albinsunnanbo]

What Authservices-version are you running?
The newly released v0.14 or an older version?

[dahlsailrunner]

Looks like v0.13.0 -- should I upgrade first then try it again?

[albinsunnanbo]

I don't expect it to make any difference, but we won't try to fix anything that can't be reproduced in the latest version.
The main reason I ask is to be sure to look in the right version of the code.
I'm curios about what things that could be null.
However the stack trace is not really useful. Looks like we don't deploy pdb files. I made an issue for that, #320, but until we get that sorted out you can check out and build the tag for the version you are using and get the pdb-files from there to get better stack traces.

[dahlsailrunner]

I'll try doing this tomorrow at the latest. Thanks for the reply.

[albinsunnanbo]

It could be something missing in the SAML2 response. You can check with https://addons.mozilla.org/sv-se/firefox/addon/saml-tracer/ to see if you get any assertions.
You can compare with https://en.wikipedia.org/wiki/SAML_2.0 , many tags are optional, but if it looks too empty it might be a clue.

[dahlsailrunner]

I think I've got some good info for you -- hoping it helps us track this down, at any rate.

With the PDB's, I was able to get a more detailed error -- it's an Object Ref problem on line 27 of AuthenticationHandler.cs.

Line 25: 
Line 26:             var authProperties = (AuthenticationProperties)result.RelayData ?? new AuthenticationProperties();
Line 27:             authProperties.RedirectUri = result.Location.OriginalString;
Line 28: 
Line 29:             return new MultipleIdentityAuthenticationTicket(identities, authProperties);

I'm not sure if it was result or result.Location that's the null ref, but it's one of those two almost certainly. I also grabbed a saml trace using the saml-tracer you suggested. (I had to just choose an image for this since I couldn't upload a text file -- the SAML tab looks like it's got good stuff in it, but I'm less familiar about what to look for here -- if there's something specific you want to see, let me know).

Any thoughts based on this new info?

Thanks a ton -
Erik

[AndersAbel]

Thanks @dahlsailrunner for your detailed error description.

Looks like this might be the same as #248, but I'm not sure. Something is obviously not quite right with the Owin ReturnUrl handling.

[dahlsailrunner]

Hmmm. I was looking at the code a bit, and discovered that in the AcsCommand file the returned CommandResult can set the location to the SPOptions.ReturnUrl, which I can configure inside my Startup.cs within my IdSrv implementation. Maybe I should set that to something? Ultimately, I need to POST an id_token to my application's login page which will send the user on their way -- but the id_token needs to include custom claims that my UserService will retrieve, so I think my redirectUrl would have to be someplace in my identity server where I can take the Okta identity that you've authenticated and then do my thing with it -- basically my UserService's AuthenticateExternalAsync. Does this sound like it might keep us moving? Any new thoughts or is it still something needing further review? Maybe I need to create some kind of custom endpoint in IdServer?

[dahlsailrunner]

I got something working! If you have a sec, I'd really appreciate you telling me if this is appropriate or not - I fear something like the nonce or state might not be correct. If the user is initiating a login request FROM THEIR IdP, they must be already logged in to it. Knowing that, I made IdSrv a "client" and constructed an Oauth2 client url that uses the okta idp. So when Kentor redirects to THAT url, it attempts to sign the user in from okta -- which they are already logged into, so it gets back into the Id Server and my custom user service and they get redirected to my application site.

The code below does work, but I'm wondering since that is in Startup -- if I'm getting a more persistent nonce and state than I would really like.

Any thoughts on this? Once I get something squared away, I can update the documentation as I did before.

This code is in the Startup.cs for my IdentityServer3.

string scopesForAuth = "openid profile roles nwpscope apibrowser";            
var state = Guid.NewGuid().ToString("N");
var nonce = Guid.NewGuid().ToString("N");
var client = new OAuth2Client(new Uri("https://iddev.nwpsmart.com/identity/connect/authorize"));
var returnUrlForOkta = client.CreateAuthorizeUrl("nwpsmartuser", "id_token token", scopesForAuth, "https://wwwdev.nwpsmart.com/Portal/Pages/Login.aspx", state, nonce, acrValues: "idp:okta", responseMode: "form_post");            
var authServicesOptions = new KentorAuthServicesAuthenticationOptions(false)
{                           
    SPOptions = new SPOptions
    {
         //EntityId = new EntityId("https://idsat.nwpsmart.com/identity/AuthServices")
         EntityId = new EntityId("https://iddev.nwpsmart.com/identity/AuthServices"),
         ReturnUrl = new Uri(returnUrlForOkta)
    },
    SignInAsAuthenticationType = signInAsType,
    AuthenticationType = "okta",
    Caption = "Okta",                               
};            

[AndersAbel]

the returned CommandResult can set the location to the SPOptions.ReturnUrl

Yes, when doing unsolicited authn responses (i.e. idp initiatied), the SPOptions.ReturnUrl must be set, as that is where the user is redirected after successful authentication.

I think that the flow is working and that you've overcome most obstacles, but the concern that the nonce and state is reused is valid. I think it opens up for a XSRF attack. If you haven't seen it, you might also want to look IdentityServer/IdentityServer3#833 and see what they have done.

[dahlsailrunner]

I think I've got the right answer now - thanks to your help and the other post you referred me to. I had seen that before but not really put two and two together until this morning.
I create a new secure page on my application site -- the RP, and then set the SPOptions.ReturnUrl to that: https://myapp.com/redirectforidplogin.aspx?idp=okta -- which then just turns around and creates the "authorizeUrl" for Id Server using okta as the idp, redirects again, and voila! Successful login with a good nonce and state value.

Woo hoo!
I've forked the AuthServices repo and will update the documentation I already wrote regarding Okta to include this idp-initiated login flow and submit a PR. But in the meantime everything seems to be working just fine!
Thanks again for your help.

[albinsunnanbo]

@AndersAbel excellent work with the improved error message in ef5c5a6
And the c#6 rewrite in AcsCommand makes it really compact.

[AndersAbel]

@albinsunnanbo I think that more error messages could benefit from a rewrite. I read a blog post some time ago that said that exception messages should be written for a developer - not for an end user. So see this as one of my first attempts to use that approach.

[dahlsailrunner]

Hi Anders and Albin - I just created a PR for updated documentation to get this all working. I also have a diagram that I could include if you think it's worth it. For now, though maybe just the text and code I thought. Let me know what you think.

[saurabhrmq]

Hi,
I am getting error as: "External login error: provider requested {0} is not a configured external provider" in Id3 AuthenticationController.
I am trying Idp initiated flow from my SampleWebForm (yes .aspx because https://github.com/Sustainsys/Saml2/blob/master/doc/IdentityServer3Okta.md using the aspx redirection); From there I put below configuration and called my local stubidp(Kentor.AuthServices.StubIdp hosted as extstub.com)

public partial class Startup {
    public void ConfigureAuth(IAppBuilder app)
    {
        app.CreatePerOwinContext(ApplicationDbContext.Create);
        app.CreatePerOwinContext<ApplicationUserManager>(ApplicationUserManager.Create);
        app.CreatePerOwinContext<ApplicationSignInManager>(ApplicationSignInManager.Create);

        AntiForgeryConfig.UniqueClaimTypeIdentifier = IdentityServer3.Core.Constants.ClaimTypes.Subject;
        JwtSecurityTokenHandler.InboundClaimTypeMap = new Dictionary<string, string>();

        app.Map("/identity", idsrvApp =>
        {
            idsrvApp.UseIdentityServer(new IdentityServerOptions
            {
                SiteName = "Embedded IdentityServer",
                SigningCertificate = new X509Certificate2(string.Format(@"{0}\bin\identityServer\idsrv3test.pfx", AppDomain.CurrentDomain.BaseDirectory), "idsrv3test"),

                Factory = new IdentityServerServiceFactory()
                                                        .UseInMemoryUsers(Users.Get())
                                                        .UseInMemoryClients(Clients.Get())
                                                        .UseInMemoryScopes(Scopes.Get()),

                AuthenticationOptions = new IdentityServer3.Core.Configuration.AuthenticationOptions
                {
                    EnableAutoCallbackForFederatedSignout = true,
                    EnablePostSignOutAutoRedirect = true,
                    //IdentityProviders = ConfigureSaml2          // I could be wrong on this line
                }
            });
        });

        app.UseCookieAuthentication(new CookieAuthenticationOptions
        {
            AuthenticationType = DefaultAuthenticationTypes.ApplicationCookie,
            LoginPath = new PathString("/Account/Login"),
            Provider = new CookieAuthenticationProvider
            {
                OnValidateIdentity = SecurityStampValidator.OnValidateIdentity<ApplicationUserManager, ApplicationUser>(
                    validateInterval: TimeSpan.FromMinutes(30), regenerateIdentity: (manager, user) => user.GenerateUserIdentityAsync(manager))
            }
        });

        app.UseExternalSignInCookie(DefaultAuthenticationTypes.ExternalCookie);

        KentorAuthServicesAuthenticationOptions authServicesOptions = new KentorAuthServicesAuthenticationOptions(false)
        {
            AuthenticationMode = Microsoft.Owin.Security.AuthenticationMode.Passive,
            AuthenticationType = "KentorAuthServices",
            Caption= "Kentor SAML"
        };
        authServicesOptions.SPOptions = new SPOptions
        {
            EntityId = new EntityId("http://localhost:44320/identity/AuthServices"),
            ReturnUrl = new Uri("https://localhost:44320/Idp_InitiatedRedirect.aspx?idp=KentorAuthServices"),
        };
        authServicesOptions.SPOptions.ServiceCertificates.Add(new X509Certificate2(AppDomain.CurrentDomain.SetupInformation.ApplicationBase + "/App_Data/Kentor.AuthServices.Tests.pfx"));

        authServicesOptions.IdentityProviders.Add(new IdentityProvider(new EntityId("https://extstub.com/Metadata"), authServicesOptions.SPOptions)
        {
            LoadMetadata = true,
            MetadataLocation = "https://extstub.com/Metadata",
            AllowUnsolicitedAuthnResponse = true
        });
        app.UseKentorAuthServicesAuthentication(authServicesOptions);
        app.UseKentorOwinCookieSaver();
        app.UseTwoFactorSignInCookie(DefaultAuthenticationTypes.TwoFactorCookie, TimeSpan.FromMinutes(5));
    }

And in IdP_InitiatedRedirect.aspx.cs
protected void Page_Load(object sender, EventArgs e)
{
var idp = Request.QueryString["idp"];
if (string.IsNullOrEmpty(idp)) throw new Exception("No idp included in redirect querystring!!");

        var scopesForAuth = "openid profile roles sampleApi";
        var state = Guid.NewGuid().ToString("N");
        var nonce = Guid.NewGuid().ToString("N");
        var client = new OAuth2Client(new Uri("https://localhost:44320/identity" + "/connect/authorize"));

        var returnUrlForOkta = client.CreateAuthorizeUrl("webform", "id_token token", scopesForAuth, "https://localhost:44320/", 
            state, nonce, acrValues: "idp:KentorAuthServices", responseMode: "form_post");

        Response.Redirect(returnUrlForOkta, false);
    }

I do see Kentor. cookie hence till Id3->signInMessage.IdP.IsPresent() is fine but context.Authentication.GetAuthenticationTypes() does not giving KentorAuthServices; not able to understand how to change to "KentorAuthServices". pls help.

[dahlsailrunner]

It might be in your client configuration within id server. can you provide Clients.cs code here? You might have limited the idp's to idsrv or something. Your code above looks fine to me but obviously is not wrking for some reason. A fiddler trace might help point you in the right direction to -- namely where the issue might be rooted.

[saurabhrmq]

thanks for reply; my client.cs
new Client
{
ClientName = "WebForm Client",
ClientId = "webform",
Flow = Flows.Implicit,
RedirectUris = new List
{
"https://localhost:44320/"
},
PostLogoutRedirectUris = new List
{
"https://localhost:44320/"
},
AllowedScopes = new List
{
"openid",
"profile",
"roles",
"sampleApi"
}
},
And I have a user as new InMemoryUser
{ Username = "JohnDoe", Password = "password", Subject = Guid.NewGuid().ToString() }
same as in default.json; so that first my webform shows Default.aspx->IdP_InitiatedRedirect.aspx->kentor stub login page->select JohnDoe user(here relayState & InResponseTo are blank)->Login->Id3 AuthenticationController->LoginExternalAsync->Invalid External Idp configured error->Render Unexpected error;
I also have noticed that in Idp flow my Unsolicited Saml response does not have "InResponseTo" but in SP flow Saml Response does contain the "InResponseTo" as guid. Since I have to complete it for my customer hence I am also looking at https://github.com/flytzen/IdPInitiatedSSOWithIdentityServer & https://github.com/scottlance/IdentityServer3Plugin; Please guide me .

[dahlsailrunner]

If you're still having trouble, I suggest getting the identity server logs going so that you can review in more detail what it's complaining about -- namely which identity provider is it saying has not been configured? A fiddler or f12 network trace would also likely help -- to see what url and query string are being passed into identity server when it fails.

And have you been able to sign in via Okta using a non-idp-initiated flow? I'm no saml expert here but I think the error is more related to the open-id connect handshakes that are trying to take place.

[saurabhrmq]

Thanks for reply Dahls; Yes I am still facing the same issue; i already mentioned in my prev comment about the missing Idp as "context.Authentication.GetAuthenticationTypes() does not giving KentorAuthServices"; so I went inside and found that the IdP_InitiatedRedirect.aspx does not create unsolicitated response for okta; the request goes as below:::
Kentor.InvokeAsync->IdP_InitiatedRedirect.aspx(Page_Load)->Kentor.ApplyResponseGrant->ApplyResponseChallenge(302)->
Id3 UseIdentityServerExtensions methods --->

Id3RequireSSLMiddleWare->ConfigureRequestId->Configure Id3 BaseUrl->ConfigureIdentityServerIssuer->ConfigureRenderLoggedOutPage->UseAutofacMiddleware->ConfigureCookieAuthentication->ConfigureSignOutMessageCookie

--->AuthorizeEndPoint Get->ProcessRequestAsync->ValidateAsync->Login->Found KentorAuthServices as signInMessage.IdP.IsPresent()->LoginExternal() and here context.Authentication.GetAuthenticationTypes() gives idsrv, idsrv.external, idsrv.partial in list; also I can see 3(sometimes 4) cookies now as SignInMessage.xxxxxxxx-> and then RenderError "UnExpected error occured"

Hence I am sure I am doing basic mistake as ApplyResponseChallenge(302) in kentor should be called as 401 then only saml response would get generated. it is aspnetidentity stuff (to make a aspx page as authorize) which I am missing.

For Okta I didnot try non-idp flow (which SP flow); bcoz first the response itself should get generated and yes you are correct in "error is more related to the open-id connect handshakes"; becoz I am also trying Id3 Mvc Auth sample for Idp flow (just Kentor stub as Idp) and there it reaches till Id3 Login and then it finds var authResult = preAuthContext.AuthenticateResult as somer user info and then RenderLoginPage() ; so I see Id3 Login page with my Kentor as external auth; if you have some pointer pls share. Thanks