| Wiki Markup |
|---|
h2. Authentication & Authorization for Sprint 2 This section reflects a revision of the original problem and it's investigation, captured below for historical purposes. h2. Goal: Provide authentication and authorization for users of the Platform, who may come in (1) through the GWT web UI or (2) from the R command line. h2. Approach:\\ Authentication for the Platform # Set-up Crowd running HTTPS. # Add users to a directory linked to Crowd, or to the Crowd database itself. # Add an authentication service, accessed via HTTPS. This service ‘talks’ to Crowd via its REST API over HTTP. The methods are: ## Get session token. (input: user name, password) ### \- Request: URL: /authenicate/v1/session user & pw: go in body method = POST \- Response: if credentials are valid: status=201, response body has display name and session token ### if credentials are invalid: status=401, header contains [https://...../authenticate/v1|https://...../authenticate/v1]/session ### plain text body: "Unable to authenticate." ## -Refresh token (input: token)- Proposal: There is no explicit 'refresh' command, rather the use of the token in service requests will refresh it. ## Invalidate token: Token goes in body. 204 response # Add a filter that applies to all Repository Services calls, which validates the token, receiving the validated user ID in the response.Service call: \-Request Header: Crowd-Session-Token: asdjfskfsdfsf \-Response If token can't be validated: \\ \\ 401 status code \\ \\ Header: WWW-Authenticate: authenticate Crowd Crowd-Authentication-Service: [https://...../authenticate/v1|https://...../authenticate/v1]\\ \\ \\ plain text body: "The token provided was invalid or expired." \\ \\ If token CAN be validated: \\ \\ Response from service is forwarded to client. Sequence of events: !04022011 Authentication.jpg|border=1!\\ Account creation (????) Can we do email verification? Authorization for the Platform # Define groups, such as project teams, in Crowd. # Copy groups to persistence layer and relate resources (e.g. datasets) to groups. # Possibly ‘mirror’ group membership from Crowd to persistence layer. User Experience: 1) Create credentials in the Platform 2) Use existing credentials 3) Use existing credentials linked to Google Apps -- on -- private -- domain. \\ Implications/requirements \- What kind of account can the user have? \- Do you get single sign on for Google Apps? \- Static IP address (can specify a host name in Crowd) h2. Crowd on Amazon Elastic Compute Cloud AWS notes: \- created an 'm1.small' \- created an EBS volume in the same zone as the server, 'us-east-1c' Followed instructions to attach volume:[http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?using-ebs.html] My server is in made new volume, in the right 'zone', vol-e47e198c In 'attach volume' dialog, specified '/dev/sdh'. Nicole's advice is: in the AWS console choose a mount point of /dev/sdh log onto the box and run sudo mount /dev/xvdh /mnt/myNewExternalDrive (The dialog says "Linux Devices: /dev/sdf through /dev/sdp") In PuTTY (following [http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?using-ebs.html|http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?using-ebs.html]) yes \| sudo /sbin/mkfs \-t ext3 /dev/sdh sudo mkdir /mnt/ebs sudo mount /dev/sdh /mnt/ebs \\ downloaded atlassian-crowd-2.1.1.tar.gz i.e. the Unix distribution and then uploaded it to /home/ec2-user/ It takes many minutes to upload. (I first tried uploading directly to /mnt/ebs but it didn't work.) sudo cp'd the file to /mnt/ebs Nicole's advice: sudo chown \-R ec2-user:ec2-user ebs/ gzip \-d atlassian-crowd-2.1.1.tar.gz tar \-xvf atlassian-crowd-2.1.1.tar so the file structure is now: /mnt/ebs/atlassian-crowd-2.1.1 set crowd.home in the crowd-init.properties file to /mnt/ebs/crowd-home \*\* will use the evaluation database for now \*\* cd'd to /mnt/ebs/atlassian-crowd-2.1.1 and typed ./start_crowd.sh it seems to have started up Also note: I haven't enabled HTTPS yet\! Need to go through these instructions: [http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL|http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL] Now go to [http://ec2-50-17-14-80.compute-1.amazonaws.com:8095|http://ec2-50-17-14-80.compute-1.amazonaws.com:8095/] It works\! I then went back to the EC2 console and STOPPED the server i-c31579af Note: I did not run the Crowd shut down script first\! \\ Server ID is BCT7-MZ9X-K8I8-PM3U generated a demo license and installed for database, chosen 'embedded' (i.e. demo version) For 'password encryption' I chose 'atlassian security'. For email server I used smtp.gmail.com with bruce.hoff and my pw. I left SSL off (which may be wrong). I bypassed setting up the OpenID integration. DONE\! [http://ec2-50-17-14-80.compute-1.amazonaws.com:8095/crowd/console/login.action|http://ec2-50-17-14-80.compute-1.amazonaws.com:8095/crowd/console/login.action]\\ Next steps: 1) Restart and see if I can log in. (May have new server name; may have to restart tomcat.) 2) Enable HTTPS (SSL). Restarted server initially there is no public DNS after a few minutes it's running, with the public DNS name: ec2-50-16-44-152.compute-1.amazonaws.com totally different from before\! Tried going to[http://ec2-50-16-44-152.compute-1.amazonaws.com:8095/crowd/console] but it failed PuTTY'd into the server: Ran 'ps', but it showed no Java process running, so I think the server is down. cd /mnt/ebs successfully, so the ebs seems to still be running. However, there are no files there and 'df' doesn't show the storage. So I think the mount was lost. The AWS console shows that the EBS volume is 'in use' and attached to the instance (i-c31579af:/dev/sdh) Back in PuTTY: sudo mount /dev/sdh /mnt/ebs Now I see the 10GB of space, but the content is gone(\!) Tried detaching and reattaching in the AWS console. Didn't work. I give up\! Started over, using my sage credentials \- created "Security Group" in the AWS console with HTTP port 8095 and HTTPS port 8443. \- created a 'micro' instance "Public DNS" is: ec2-50-17-17-19.compute-1.amazonaws.com copied atlassian-crowd-2.1.1.tar.gz to /home/ec2-user gunzip atlassian-crowd-2.1.1.tar.gz sudo mv atlassian-crowd-2.1.1.tar /usr/local sudo tar \-xvf atlassian-crowd-2.1.1.tar sudo chmod \-R 777 atlassian-crowd-2.1.1 edited crowd-webapp/WEB-INF/classes/crowd-init.properties just uncommented the default so crowd.home=/var/crowd-home ./start_crowd.sh [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd] resulted in 404 error. So tomcat is running (and listening on port 8095), but Crowd isn't My guess is that it's a permissions problem. sudo chmod \-R 777 var stopped/restarted Crowd Now it works\! Went through the rest of the process of setting up the server. need to set up the application key/pw pair. Created 'Generic Application' with user/pw: platform/platform-pw For the URL I set:[http://deflaux-test.appspot.com/] Note that this is NOT https\! 'Resolve IP Address' generated 127.0.0.1, which is curious... Checked "Allow all users to authenticate." Created a user called 'demouser' with password 'demouser-pw' Created a group called 'demogroup'. Create a token C:\Users\bhoff>curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>demouser </username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica tion-context>" \-X POST [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true] \* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0) \* Trying 50.17.17.19... connected \* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0) \* Server auth using Basic with user 'platform' > POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1 > Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc= > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095 > Accept: application/xml > Content-Type: application/xml > Content-Length: 295 > < HTTP/1.1 403 Forbidden < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.1 < Content-Type: text/html;charset=utf-8 < Content-Length: 1423 < Date: Wed, 09 Feb 2011 23:37:14 GMT < <html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><\!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans \-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;col or:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {c olor : black;}A.name {color : black;}HR {color : #525D76;}\--></style> </head><body><h1>HTTP Status 403 - Client with address "140.107.149.249", and hostname "140.107.149.249" is fo rbidden from making requests to the application, platform.</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Client with address "140.107.149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, platform.</u></p><p><b>description</b> <u>Access to the specified resource (Client with address "140.107 .149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, platform.) has been forbidden.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6. 0.26</h3></body></html>\* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact \* Closing connection #0 Added the remote address. Now: C:\Users\bhoff>curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>demouser </username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica tion-context>" \-X POST [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true] \* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0) \* Trying 50.17.17.19... connected \* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0) \* Server auth using Basic with user 'platform' > POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1 > Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc= > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095 > Accept: application/xml > Content-Type: application/xml > Content-Length: 295 > < HTTP/1.1 201 Created < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.1 < Set-Cookie: JSESSIONID=07302C98E4992295F04F0FA49BA44B05; Path=/crowd < Cache-Control: no-cache, no-store, no-transform < Location: [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMqaRBLAyQ00] < Content-Type: application/xml < Content-Length: 422 < Date: Wed, 09 Feb 2011 23:40:10 GMT < <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><session expand="user"><token>OcdmfMhMTeEUmMqaRBLAyQ00</token><user name="demouser"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazo naws.com:8095/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMq aRBLAyQ00"/></session>\* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact \* Closing connection #0 Now to get the details for the user: C:\Users\bhoff>curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-k \-X GET [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/us] er?username=demouser <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><user name="demouser" expand="attributes"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/ user?username=demouser"/><first-name>demo</first-name><last-name>user</last-name><display-name>demo user</display-name><email>bruce.hoff@sagebase.org</email><password><link rel="edit" href="http://ec2 \-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/password?username=demouser"/></password><active>true</active><attributes><link rel="self" href="http://ec2-50-17-17-19.comput e-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/attribute?username=demouser"/></attributes></user> Now to try running from the web: changed the version in appengine-web.xml to 'repoAndAuth' then ran mvn gae:deploy from projects/trunk/services Result: 'build failure' mvn gae:run doesn't work either needed to run: 'mvn gae:unpack' Now, when I 'mvn gae:run' I can test from localhost: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://localhost:8080/repo/v1/session] {"displayName":"demo user","token":"00Cqwht6MS7mmyEiqey2Bw00"} yeah\! Next step deploy: set deployment to version 'auth' on deflaux-test.appspot.com ran: mvn gae:deploy curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://auth.latest.deflaux-test.appspot.com/repo/v1/session] got error, probably because Crowd isn't accepting messages from the app ping auth.latest.deflaux-test.appspot.com Pinging appspot.l.google.com \[74.125.127.141\] ... Added 74.125.127.141 to allowed list for 'platform' in Crowd. Still didn't work. Added more debug info, and found that this is the rejected IP: 74.125.75.17 Added the IP to the allowed list. It works\!\! curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [https://auth.latest.deflaux-test.appspot.com/repo/v1/session] Hitting GAE via https also works: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-k \-X POST [https://auth.latest.deflaux-test.appspot.com/repo/v1/session] \!\!\!\! Now to create a Crowd server that talks SSL: Created another micro instance ec2-75-101-179-108.compute-1.amazonaws.com and followed the earlier instructions to set up Crowd. BEFORE starting it up, I followed the instructions for enabling SSL:[http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL] I tweaked the Connector element in the apache-tomcat/conf/server.xml file, to add three more attributes, seen in the existing Connector element. <Connector port="8443" maxHttpHeaderSize="8192" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" enableLookups="false" disableUploadTimeout="true" acceptCount="100" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" SSLEnabled="true" keystoreFile="/home/ec2-user/.keystore" keystorePass="changeit" connectionTimeout="20000" disableUploadTimeout="true" useBodyEncodingForURI="true"/> Ran keytool \-genkey \-alias tomcat \-keyalg RSA \-sigalg MD5withRSA which creates /home/ec2-user/.keystore and I also copied the .keystore file to my desktop ('cause I think I'll need to stick it in my appengine application). Edited crowd.properties: The instructions say to edit <Crowd-Home-Directory>/crowd.properties i.e. /var/crowd-home/crowd.properties but this directory doesn't exist yet. I think it's created upon start up. An educated guess is to edit it here: /usr/local/atlassian-crowd-2.1.1/etc/crowd.properties with the expectation that it will ultimately be moved to /var/crowd-home. But looking at this file I see only " (CROWD)", not the protocol specific URLs. Made apache-tomcat/bin/setenv.sh look like JAVA_OPTS="-Xms128m \-Xmx512m \-XX:MaxPermSize=256m \-Dfile.encoding=UTF-8 $JAVA_OPTS \-Djavax.net.ssl.keyStore=/home/ec2-user/.keystore \-Djavax.net.ssl.keyStorePassword=changeit \-Djavax.net.ssl.trustStore=/home/ec2-user/.keystore \-Djavax.net.ssl.trustStorePassword=changeit" export JAVA_OPTS Ran /user/local/atlassian-crowd-2.1.1/start_crowd.sh but I don't see /var/crowd-home ps doesn't show any Java process running. One problem is that I entered disableUploadTimeout="true" twice in the connector. After removing the duplicate and running 'start_crowd.sh' again, ps shows a Java process running. Still hasn't made /var/crowd-home but log files don't show any errors. Try going to [https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/console] I had to go through the 'untrusted certificate' warning in Firefox, but it works. Went through the rest of the server set up. Note: The settings include a max number of days 'til the user has to change her password. For now I'm not using this. Added demouser, demouser-pw Added application with user/pw: platform/platform-pw Now let's try to generate a session from curl: curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>demouser</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentication-context>" \-X POST [https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session?validate-password=true] Got the error saying I have to add 140.107.149.249 After adding this address to the 'allowed' list, it works. The response is: <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><session expand="user"><token>2ffROYdzaeThREyIPbV7Wg00</token><user name="demouser"><link rel="self" href="https://ec2-75-101-179-108.compute-1.a mazonaws.com:8443/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session/2ffROYd zaeThREyIPbV7Wg00"/></session> Tried running Java app. It fails because I don't have the necessary certificate in Java's keystore. The documentation seems to indicate that this isn't a problem for appengine's 'fetch' service. Now try to hit the service, running locally: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://localhost:8080/repo/v1/session] ooops it's changed: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"userId\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://localhost:8080/repo/v1/session] Didn't work: fetch service runs but prints message: java.io.IOException: Could not fetch URL: [https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/latest/session?validate-password=true] Could it be that my IP address has changed? It is 140.107.177.53 Yes, that's new\! I added it and reran the request: ... with the same result. Just for fun, try deploying to a new app version called 'ssl'. The test is now: curl \-k \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"userId\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [https://ssl.latest.deflaux-test.appspot.com/repo/v1/session] fails :^( Looking at the stack trace in the GAE console, it's rejecting the IP address 74.125.66.80. Added the address in the Crowd application console and reran: fails :^( hey, the address changed again, this time to: 74.125.75.1 Added this address .... it worked\!\! C:\Users\bhoff\eclipse projects\trunk\services\repository>curl \-k \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "\{\"userId\":\"demouser\", \"pw\":\"demouser-pw\"\}" \-X POST https ://ssl.latest.deflaux-test.appspot.com/repo/v1/session \{"displayName":"Demo User","token":"ih9byQiH0m80PbxKY10Byw00"\} Following some advice found on-line, got the desktop application to work as well. [http://www.exampledepot.com/egs/javax.net.ssl/trustall.html] [http://stackoverflow.com/questions/2186543/java-secure-webservice] h2. ---- h2. Original Problem Formulation: h2. Goal: Provide authentication / authorization for users of the Sage Platform. Authentication: Verify user's identity. Authorization: Allow a user, or an application invoked by user, to access data in the Platform. \\ The Platform comprises: \- Addama registry (running on Google App Engine, or "GAE") \- Addama feed service (on GAE) \- Addama file repository service (running on Amazon Elastic Compute Cloud, or "EC2") \- Addama Java Content Repository (JCR) service (on EC2) \- UI html files (on EC2) \- a Google Group \- shared Google Docs \- file repository, hosted at Sage, accessible via Secure CoPy (SCP) \\ h2. Requirements and Design Constraints: \- Single sign-on. Once a user has signed on to the platform, they don't have to sign into any of the components. \- One-stop user administration: Adding or removing a user in one place will apply to all components. \- "Security at all layers": No component can be part of the platform unless it adheres to (one of) our authentication mechanism(s). (Note, we could implement several mechanisms "under the hood", if the systems we are integrating require it.) \- Platform will have 'arms length' integration with Google Apps, Groups. (I.e. the rest of the system (Addama, Sage file repository) must work if Google tools are omitted.) \- We want to have full control over the UI (hence a custom approach using GWT instead of GoogleSites) but link to the relevant GoogleDocs and GoogleGroups and use the GoogleDocs UI and GoogleGroups UI when folks are interacting with those resources. \\ h2. Use Cases \- create a google doc, share with the group. \- send a message to the group \- create an analysis, share with the group \- add a user to the platform \- add all users from a collaborating lab to the platform \- create user credentials for a new user, providing access to the platform \- give a user or a group of users access to an addama service h2. Analysis There are just four components that need to perform authentication. (The others delegate authentication to the registry.) They are listed here, along with authentication options: Addama registry GAE application (Google account, Google Apps account, OpenID federated authentication) Google Apps (Google Apps account, SAML delegated authentication) Google Group (Google account, Google Apps account) Sage SSH server (standard unix login) If the SSH server were eliminated (by migrating the hosted files to an Addama repository service) then a common denominator \*might\* be Google App account authentication, which in turn might be delegated to an external identity provider. h2. Design This is a possible design approach, contingent upon the answers to certain open questions (listed below): \- Restrict the Addama registry application to be hosted on a proprietary domain, configured to authenticate via "Google Apps for your domain". \- Configure Google Apps on our domain to delegate authentication via SAML. \- Configure Google Groups on our domain to delegate authentication via SAML. \- Migrate local file repository to Addama service. \- Employ Atlassian Crowd as the administration console for user authentication. h3. Open Questions \- Are Atlassians Crowd pricing, license models, and hosting options acceptable for our purposes? Do they prohibit integrating with NextBio? (Note: Atlassian doesn't host Crowd, rather we download and host it ourselves. It's an Apache Tomcat application, with a variety of choices for databases.) \- What other SAML or OpenID identity provider (ip) tools (provding UIs and/or aggregating other IdP's) are there? \- Can Google Apps and Google Groups use OpenID (instead of SAML) for authentication? *NO.* They will authenticate 3rd party app's via OpenID but they do not +delegate+ authentication via OpenID. \-Do we want to use google app's to see content we host elsewhere, or will google app's be the only place that doc's are stored in this 'sprint'? - Can "Google Group" membership be managed by an external authentication mechanism? *YES, it's part of the Google Apps delegation, see below.* \- if we are doing "arm's length" integration with google app's, then what other providers should we plan for? \- do we need 'audit logs', e.g. to show when users were added/removed and by whom? h4. Pivotal assumptions/questions (i.e. that can substantially change the design) \- Does a google doc created outside of the domain need to be able to be shared with a group in the domain? \- Do new users need to be given immediate access to the platform (without an admin in the loop)? \\ h3. Experiment to address key questions h4. 1. Authenticate Google Apps, Groups using SAML and Crowd \- Set-up Crowd trial edition (on local box or AWS) \- Change Google Apps demo domain to authenticate against Crowd \- Add user to Crowd \-Try to access Google Apps via this user (e.g. make a document) \- try to access Google Groups via this user h4. 2. Authenticate GAE app using SAML and Crowd \- Change/deploy GAE app, authenticating via Google Apps \- Try to log into to GAE app via this user (If not, can GAE OpenID option work with Crowd or can bypass UserService to use some sort of OpenID connector to reach Crowd?) \\ h4. 3. Authorize using SAML, Crowd \- Define a group in Crowd \- Add a user to a group in Crowd \- Add a user to a group in Google Apps \- See if access to services can be selected based on such group membership. h4. 4. Replace Crowd with Open Source Identity Provider Repeat 1-3 above. h3. Experiment execution h4. Set Up Crowd \- Added 140.107.149.214 deflaux to C:/windows/system32/drivers/etc/hosts on my la top. I can now PuTTY/SSH into 'deflaux' which is a Linux box in Nicole's office. \- Followed [http://confluence.atlassian.com/display/CROWD/Installing+Crowd+and+CrowdID|http://confluence.atlassian.com/display/CROWD/Installing+Crowd+and+CrowdID] \- downloaded zip file \- downloaded and installed WinSCP; connected to deflaux:22 using SCP protocol. \- unzipped zip file and copied contents to /usr/local/tomcat on deflaux \- per the instructions, created the directory /var/crowd-home and edited .../crowd-webapp/WEB-INF/classes/crowd-init.properties accordingly. \- ran sudo ./start_crowd.sh but got "The BASEDIR environment variable is not defined correctly. This environment variable is needed to run this program" \- Googled around for a solution. Found [http://codingexplorer.wordpress.com/2009/01/12/the-basedir-environment-variable-is-not-defined-correctly-this-environment-variable-is-needed-to-run-this-program/|http://codingexplorer.wordpress.com/2009/01/12/the-basedir-environment-variable-is-not-defined-correctly-this-environment-variable-is-needed-to-run-this-program/] \- Did a bunch of trial-and-error and ended up trying chmod \-R 777 * which seemed to do the trick: Instead of getting an error message I got: Using CATALINA_BASE: /usr/local/tomcat/apache-tomcat Using CATALINA_HOME: /usr/local/tomcat/apache-tomcat Using CATALINA_TMPDIR: /usr/local/tomcat/apache-tomcat/temp Using JRE_HOME: /usr Using CLASSPATH: /usr/local/tomcat/apache-tomcat/bin/bootstrap.jar \- Instructions say to go to [http://localhost:8095/crowd|http://localhost:8095/crowd]. I tried going to [http://deflaux:8095/crowd|http://deflaux:8095/crowd] and [http://140.107.149.214:8095/crowd|http://140.107.149.214:8095/crowd], but neither worked. Used 'sudo ps' and 'sudo lsof \-i :8095' to show that the server is indeed running. \- Nicole poked hole in the firewall on the box. Now I can go to [http://deflaux:8095/|http://deflaux:8095/] and see the web page. Woo hoo. \- Per the instructions at [http://confluence.atlassian.com/display/CROWD/Setting+Crowd+to+Run+Automatically+and+Use+an+Unprivileged+System+User+on+UNIX|http://confluence.atlassian.com/display/CROWD/Setting+Crowd+to+Run+Automatically+and+Use+an+Unprivileged+System+User+on+UNIX] I created the file /usr/local/tomcat/crowd.init.d, then from /etc/init.d sudo ln /usr/local/tomcat/crowd.init.d crowd \\ Ran the web-based set up wizard: Got a 30 day license key and chose to use the 'embedded' database. h4. Change Google Apps demo domain to authenticate against Crowd Following: [http://confluence.atlassian.com/display/CROWD/Configuring+the+Google+Apps+Connector|http://confluence.atlassian.com/display/CROWD/Configuring+the+Google+Apps+Connector] Note, this says "you will need the Premier, Education, or Partners edition of Google Apps." so I may not be able to use 42stories.com. I'll see how far I get before I'm stuck. \- Since we're using JDK 1.6, I followed the instructions which said to put the following two jars in <Crowd-Install>/crowd-webapp/WEB-INF/lib (<Crowd-Install>=/usr/local/tomcat) 1-) [xml-security-1.4.2.jar|http://people.apache.org/repo/m2-ibiblio-rsync-repository/org/apache/santuario/xmlsec/1.4.2/] 2-) [commons-logging-1.1.1.jar|http://commons.apache.org/logging/]\\ \- At step 1.5 "...select one or more user [directories|http://sagebionetworks.jira.com/display/CROWD/Using+the+Directory+Browser]..." I picked the single one listed, "Evaluation." I believe this is the default user directory I set up during installation. Set "Allow all to authenticate" \-> True. Under "Permissions" allow Google Apps to add/modify/remove groups and users, but I'm not sure if Google Apps can actually do this. (Perhaps it can, rhrough the Provisioning API\!) Step 2: Generate new keys. Afterwards the Configuration tab displayed: Sign-in Page URL: [http://deflaux:8095/crowd/console/plugin/secure/saml/samlauth.action|http://deflaux:8095/crowd/console/plugin/secure/saml/samlauth.action] Sign-out Page URL: [http://deflaux:8095/crowd/console/logoff.action|http://deflaux:8095/crowd/console/logoff.action] Change Password URL: [http://deflaux:8095/crowd/console/user/viewchangepassword.action|http://deflaux:8095/crowd/console/user/viewchangepassword.action] DSA Key-pair Location: /var/crowd-home/plugin-data/crowd-saml-plugin \- Step 3. Configuring Google Apps to Recognise Crowd Went to 42stories google app's console: [https://www.google.com/a/cpanel/42stories.com/Dashboard|https://www.google.com/a/cpanel/42stories.com/Dashboard] There is no "'*single sign-on (SSO)*' link." Switched to sagebionetworks.com, which DOES have a premier version of Google Apps. Followed Atlassian instructions to set up SSO. Note: To Disable: Go to [https://www.google.com/a/cpanel/sagebionetworks.com/SetupSSO|https://www.google.com/a/cpanel/sagebionetworks.com/SetupSSO], unclick "*Enable Single Sign-on",* then Save Changes. Step 4, trying it out: I created a user called 'ssotest' having the same password. Performed the 'Authentication Test' which was successful. Now for a true test, connecting to Google Apps on bionetworks.com using 'ssotest': Went to [http://sites.google.com/a/sagebionetworks.com|http://sites.google.com/a/sagebionetworks.com] Click on 'sign in to Sage Bionetworks' Entered ssotest / ssotest got "Google Apps - Invalid Email" error I \*can\* log in to bruce.hoff. This is because sagebionetworks already has a bruce.hoff Added a 'mike.kellen' pw: ssotest to Crowd It works\! Added 'nicole.deflaux', pw: drizzle to Crowd It works\! Conclusion: GoogleApps delegates password management, but not user management\!\! Went to groups.google.com/a/sagebionetworks.com It works\! I.e. google groups delegates authentication too. \\ *{_}Big open question: If Crowd aggregates two directories, both having a user called john_smith, then whose credentials are used to log in to Google Apps?_* \\ Tried running Nicole's demo. Result: Was prompted for regular (non-Crowd) credentials. So this demo doesn't automatically delegate when google apps does. This might be due to how the application was deployed. The application is associated with the sagebase.org domain, i.e. it is visible at: [https://appengine.google.com/a/sagebase.org|https://appengine.google.com/a/sagebase.org]\\ at the authentication choice is "Google Accounts API: The [Google Accounts API|http://code.google.com/appengine/docs/users.html] includes all Gmail Accounts, but does not include accounts on Google Apps domains." Info on how to deploy to a domain is here: [http://code.google.com/appengine/articles/auth.html|http://code.google.com/appengine/articles/auth.html] h4. Create a Google App Engine application using Google Apps accounts to log-in Installed GAE plug-in for Eclipse. It includes SDK v. 1.3.8. Created an app 'sandbox-sagebionetworks.appspot.com' set to authenticate against users in the sagebionetworks.com domain. Verified that the default app runs on the web. Added a <security-constraint> to the web.xml and redeployed. Result: I get a "500 Internal Server Error" error when I click on the servlet link. In the appengine control panel error log I see the message: "Authentication for the Google Apps domain sagebionetworks.com can only be performed when requests are served from a subdomain of that domain or it has been approved through the Google Apps Control Panel." I logged in using my sagebionetworks.com credentials, but got the same error again. The problem is that I have not told GAE to delegate to Google Apps. Went to appengine.google.com under the sandbox-sagebionetworks app clicked 'Application Settings' then went to 'Add Domain' entered 'sagebionetworks.com got the message: Your users can access sandbox-sagebionetworks at: [https://sandbox-sagebionetworks.appspot.com|https://sandbox-sagebionetworks.appspot.com/] Now it works\! I can go to [https://sandbox-sagebionetworks.appspot.com/|https://sandbox-sagebionetworks.appspot.com/], click on the link, and get "Hello, world" I am signed in as [bruce.hoff@sagebionetworks.com|mailto:bruce.hoff@sagebionetworks.com]. I click 'Sign Out' from Google Apps and try the app url again. Unexpectedly I CAN get to 'hello, world' (no authentication) I close all windows I try to go to google.com/a/sagebionetworks.com and am prompted for a log-in. Now I go to the app, click on the "Sandbox" link and am prompted for a log-in. (Yea\!) Logged out of Google Apps [https://www.google.com/a/sagebionetworks.com/|https://www.google.com/a/sagebionetworks.com/] then returned to the app and was prompted for a log-in, so authentication seems to be working. Now to delegate authentication to Crowd: Logged in to the control panel for the sagebionetworks.com domain and went to 'advanced tools' [https://www.google.com/a/cpanel/sagebionetworks.com/Advanced#Advanced/subtab=0|https://www.google.com/a/cpanel/sagebionetworks.com/Advanced#Advanced/subtab=0] Went to 'set up single sign-on (SSO)" [https://www.google.com/a/cpanel/sagebionetworks.com/SetupSSO|https://www.google.com/a/cpanel/sagebionetworks.com/SetupSSO] and clicked "Enable Single Sign-on" then "Save changes" Logged out. Went to sites.google.com/a/sagebionetworks.com and got the modified Google log in reflecting that SSO is activated. Clicked "Sign in to Sage Bionetworks" and got the Crowd log in screen. Did NOT log in but rather went to [https://sandbox-sagebionetworks.appspot.com|https://sandbox-sagebionetworks.appspot.com/] Clicked on 'Sandbox' and went to the Crowd log-in screen\!\! (Success\!\!) Entered user: nicole.deflaux, p/w: drizzle (avoiding my own, administrative credentials) Successfully ran the "Hello world" servlet. Summary: Google App Engine (GAE) can be configured to delegate authentication to Google Apps (on our domain), which can in turn delegate authentication to an external SAML-based Identity Provider. Moreover, the authentication requirement for the GAE services can be completely managed in the web.xml file using <security-constraint> tags. In principle we can create a collaborative platform including Google Apps, Google Groups, and Google App Engine authenticating web services (e.g. Addama) in which users experience single-sign on using their native credentials and externally managed passwords. h4. Authorize using SAML, Crowd \- Define a group in Crowd: Crowd defines the 'crowd-administrators' group. \- Add a user to a group in Crowd: the user 'bruce.hoff' is in the 'crowd-administrators' group. \- Add a user to a group in Google Apps: there is a group 'demo' having several members. \- See if access to services can be selected based on such group membership. Note: Crowd does not see the google group 'demo'; Google Apps does not see the Crowd group 'crowd-administrators. This suggests that the group definition component of authorization is not delegated by Google Apps. The continuation of this experiment is to experiment with control of access via the Google Apps group. Assuming that Google Apps groups 'authorize' access to Google Apps (doc's, sites, group threads, etc.), then the two open questions are: 1) Can a web app (e.g. the Addama registry) perform authorization against the Google Apps groups; 2) Can group membership be controlled programmatically. (This is a minor need, since the Google Apps control panel allows administration of group membership.) h5. Connect GAE to Google Apps for group-based authorization, using 2-legged OAuth To give the GAE app access to users and groups on the sagebionetworks.com domain, I went to [https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients|https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients] and entered Client Name: sandbox-sagebionetworks.appspot.com One or more API Scopes: [https://apps-apis.google.com/a/feeds/group/#readonly|https://apps-apis.google.com/a/feeds/group/#readonly], [https://apps-apis.google.com/a/feeds/user/#readonly|https://apps-apis.google.com/a/feeds/user/#readonly] the scopes came from: [http://www.google.com/support/a/bin/answer.py?answer=162106|http://www.google.com/support/a/bin/answer.py?answer=162106] Result: Got the error: This client name has not been registered with Google yet. Followed the instructions here: [http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html|http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html] 1. Went to here to register the app: [https://www.google.com/accounts/ManageDomains|https://www.google.com/accounts/ManageDomains] 1.1 Entered [https://sandbox-sagebionetworks.appspot.com|https://sandbox-sagebionetworks.appspot.com/] 1.2 We to the "verify ownership" page and chose the "Upload an HTML file to your server" option downloaded the generated html file into the 'war' folder of my GAE app completed the 'verify ownership' step 1.3 Targe URL path prefix: [https://sandbox-sagebionetworks.appspot.com/authsub|https://sandbox-sagebionetworks.appspot.com/authsub] I did not complete the section 'upload new X.509 cert' (more at [http://code.google.com/apis/gdata/docs/auth/authsub.html|http://code.google.com/apis/gdata/docs/auth/authsub.html]) but clicked 'save'. Result: OAuth Consumer Key: sandbox-sagebionetworks.appspot.com OAuth Consumer Secret: OrB7tgg7BJtAkn9uvbo14uC9 To continue the process I need to return here: [https://www.google.com/accounts/UpdateDomain|https://www.google.com/accounts/UpdateDomain] following these instructions: [http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html|http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html] Next: Need to figure out what to put at the 'authsub' URL Went back to [https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients|https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients] and clicked 'authorize' again. This time it worked\!\! \\ Time to write the web app. On the subject of 2-legged OAuth, from [http://code.google.com/apis/accounts/docs/OAuth.html#WorkingOauth|http://code.google.com/apis/accounts/docs/OAuth.html#WorkingOauth] "This differs from the normal authorization flow, also known as 3-legged OAuth, in that no access token is required. All applications using 2-legged OAuth must be registered with Google." There's a code example here: [http://code.google.com/apis/gdata/docs/auth/oauth.html#2LeggedOAuth|http://code.google.com/apis/gdata/docs/auth/oauth.html#2LeggedOAuth] another example: [http://code.google.com/p/gdata-java-client/source/browse/trunk/java/sample/oauth/TwoLeggedOAuthExample.java|http://code.google.com/p/gdata-java-client/source/browse/trunk/java/sample/oauth/TwoLeggedOAuthExample.java] From [http://code.google.com/googleapps/domain/gdata_provisioning_api_v2.0_developers_protocol.html#Authenticating_To_Your_Domain|http://code.google.com/googleapps/domain/gdata_provisioning_api_v2.0_developers_protocol.html#Authenticating_To_Your_Domain] To retrieve all groups in a particular domain, use the following GET request: GET \[https://apps-apis.google.com/a/feeds/group/2.0/domain\[?\[start\|https://apps-apis.google.com/a/feeds/group/2.0/domain\[?[start]=\]\] To retrieve all the groups for a particular member \[in a domain\], use the following GET request: GET \[https://apps-apis.google.com/a/feeds/group/2.0/domain/?member=memberId\[&directOnly=true\|false\|https://apps-apis.google.com/a/feeds/group/2.0/domain/?member=memberId[&directOnly=true|false]\] directOnly - If true, the request identifies only members with direct association with the group. \\ Tried the first query, i.e. GET [https://apps-apis.google.com/a/feeds/group/2.0/sagebionetworks.com|https://apps-apis.google.com/a/feeds/group/2.0/sagebionetworks.com] with these request properties: conn.setRequestProperty("Authorization: OAuth", "oauth_consumer_key=sandbox-sagebionetworks.appspot.com, " +"oauth_nonce="+nonce()+", " +"oauth_signature_method=HMAC-SHA1, " +"oauth_signature=OrB7tgg7BJtAkn9uvbo14uC9" +"oauth_timestamp="+System.currentTimeMillis() ); Result: Got 401 OK :^( Added xoauth_requestor_id param in the URL Result: Got 401 OK :^( Will try Google example from here: [http://code.google.com/p/gdata-java-client/source/browse/trunk/java/sample/oauth/TwoLeggedOAuthExample.java|http://code.google.com/p/gdata-java-client/source/browse/trunk/java/sample/oauth/TwoLeggedOAuthExample.java] Went here to get the Java API [http://code.google.com/p/google-api-java-client/|http://code.google.com/p/google-api-java-client/] downloaded the file google-api-client-1.2.1-alpha-java.zip Next steps: 1. Complete the installation 2. Get the google example running Need google data java client: [http://code.google.com/apis/gdata/articles/java_client_lib.html|http://code.google.com/apis/gdata/articles/java_client_lib.html] Chose this [http://code.google.com/p/gdata-java-client/downloads/detail?name=gdata-src.java-1.42.0.zip&can=2&q|http://code.google.com/p/gdata-java-client/downloads/detail?name=gdata-src.java-1.42.0.zip&can=2&q]= extracted the file to c:/users/hoff/eclipse there are a lot of files, including source ran the project but got 'classnotfound' exceptions had to copy files to war/web-inf/lib in addition to adding them to the project as external jars got java.lang.NoClassDefFoundError: com/google/common/collect/Maps downloaded [http://code.google.com/p/gdata-samples/source/browse/trunk/sites/java/UploadApp/lib/google-collect-1.0-rc1.jar?r=193|http://code.google.com/p/gdata-samples/source/browse/trunk/sites/java/UploadApp/lib/google-collect-1.0-rc1.jar?r=193] and put directly into the war/web-inf/lib (not build path) app now runs. Fixed a couple nullptr exceptions and it ran w/o error. Restored the security-constraint to the web.xml file (accidentally erased). Deployed to GAE. It works\! I had to iterate on the code example to get the group information out, but eventually saw the two groups 'demo' and 'test' in the sagebionetworks.com domain. Next step: retrieve group membership for a user. Done: The authorization demo now runs \-\- comparing user's groups to those authorized for a service. h3. Authenticate and Authorize a web application directly against Crowd The approach taken above is 'tightly integrated' with Google Apps since any user accessing any resource (even a non-Google App resource) needs to have a Google Apps account. It would be helpful if users could authenticate against Crowd directly, even without having a Google Apps account on our domain. The next phase pursues this approach: First I tried implementing the Crowd Java Client, in a GAE application. This turned out to be a dead end, but I include my notes anyway: h4. Implementing the Crowd Java Client notes from implementing Crowd demo: 1. Moved the unzipped 'atlassian-crowd-2.1.0' under my 'eclipse projects' folder 2. Created a new java project from the folder atlassian-crowd-2.1.0/demo-src 3. Add external jar files from atlassian-crowd-2.1.0/client/ crowd-integration-client.2.1.0.jar lib/*.jar Result: There are build errors: Logoff.java won't compile, with messages about HttpServlet... missing The project is missing the J2EE stuff. I think the GAE projects enjoy having servlet libraries wrapped up with other google libraries. This seems to be the authentication API: [http://docs.atlassian.com/crowd/2.1.0-beta2/index.html?com/atlassian/crowd/integration/http/CrowdHttpAuthenticator.html|http://docs.atlassian.com/crowd/2.1.0-beta2/index.html?com/atlassian/crowd/integration/http/CrowdHttpAuthenticator.html] Looks like a way to integrate as a "Filter" [http://docs.atlassian.com/crowd/2.1.0-beta2/index.html?com/atlassian/crowd/integration/http/CrowdHttpAuthenticator.html|http://docs.atlassian.com/crowd/2.1.0-beta2/index.html?com/atlassian/crowd/integration/http/CrowdHttpAuthenticator.html] Here's a demo for getting the 'principals' for a Group: [http://docs.atlassian.com/crowd/2.1.0-beta2/com/atlassian/crowd/console/action/group/ViewGroupMembers.html|http://docs.atlassian.com/crowd/2.1.0-beta2/com/atlassian/crowd/console/action/group/ViewGroupMembers.html] It would be 'slicker' to find the groups for a principal rather than the principals for a group. \-------------------------------------\- From: [http://confluence.atlassian.com/display/CROWDDEV/Java+Integration+Libraries|http://confluence.atlassian.com/display/CROWDDEV/Java+Integration+Libraries] VerifyTokenFilter \-\- looks like what I want. To try out Crowd authentication: 1. Split the GAE demo into a 'GoogleApps' section and a 'Crowd' section 2. Added the VerifyTokenFilter to web.xml (just for the 'Crowd' secton of the app) 3. Copied eclipse projects/atlassian-crowd-2.1.0/crowd-integration-client-2.1.0.jar to war/WEB-INF/lib at this point I tried running the app but got java.lang.NoClassDefFoundError: org/apache/log4j/Logger 4. Took the hint and added all the file in atlassian-crowd-2.1.0/client/lib to war/WEB-INF/lib Tried running again, but got java.lang.NoClassDefFoundError: java.rmi.server.UID is a restricted class. Please see the Google App Engine developer's guide for more details. after VerifyTokenFilter indirectly called it. According to this [http://jira.atlassian.com/browse/CWD-1748|http://jira.atlassian.com/browse/CWD-1748] there is an additional issue, which is that the Crowd client tries to spawn threads, not allowed by GAE. So I may have to reimplement the authentication using the ('restful') web api. \\ h4. Implementing the Crowd RESTful API The documentation starts here: [http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+APIs|http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+APIs] I should try this example: [http://YOUR-CROWD-SERVER:8095/rest/usermanagement/1/user?username=USERNAME|http://your-crowd-server:8095/rest/usermanagement/1/user?username=USERNAME] [http://deflaux:8095/rest/usermanagement/1/user?username=mike.kellen|http://deflaux:8095/rest/usermanagement/1/user?username=mike.kellen] returns 404 error, probably because I have not included the application credentials in the request. From [http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+Resources|http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+Resources] This looks like the way to find if a user is a member of a group: /user/group/nested?username=USERNAME&groupname=GROUPNAME near the bottom: Crowd SSO Token Resource seems to give the methods for authenticating the user. I believe I need credentials for making the request. I logged in to the crowd admin console. I clicked 'add application' and entered the following parameters: Application Type: Generic Application name: gae-client description: gateway filter for authentication against crowd password: gae-client-pw <next page> url: [http://sandbox-sagebionetworks.appspot.com/|http://sandbox-sagebionetworks.appspot.com/] <Clicked 'Resolve IP Address', which filled in the next field:> Remote IP Address: 127.0.0.1 (I don't get this\!) <next page> Directories: Evaluation Authorisation: Clicked "Let all users in this directory authenticate against the gae-client application." \*DONE\* Tried writing a class 'CrowdAuthUtil' to isse the RESTful request to get a given user It gives me the 404 error. I have to incude the credentials. Based on my reading, I should be getting a '401' response, to challenge me for the credentials. I may have to run it on GAE to get this far. I changed 'deflaux' to '140.107.149.214' so GAE can find Crowd. Got Timeout while fetching: [http://140.107.149.214:8095/rest/usermanagement/1/user?username=mike.kellen|http://140.107.149.214:8095/rest/usermanagement/1/user?username=mike.kellen] maybe I need to host the Crowd server elsewhere\! From [http://confluence.atlassian.com/display/CROWDDEV/Using+the+Crowd+REST+APIs|http://confluence.atlassian.com/display/CROWDDEV/Using+the+Crowd+REST+APIs] Access to all resources (using any method) requires the client to be authenticated via basic authentication. See RFC 2617. [http://frontier.userland.com/stories/storyReader$2159|http://frontier.userland.com/stories/storyReader$2159] Suggests I should be getting an authentication challenge when I issue an HTTP Request. Found a Base64 encoder here: [http://sourceforge.net/projects/iharder/files/base64/2.3/|http://sourceforge.net/projects/iharder/files/base64/2.3/] and added the basic authorization property to the http request. This guy has the same problem as me: [http://forums.atlassian.com/thread.jspa?messageID=257358182|http://forums.atlassian.com/thread.jspa?messageID=257358182] Need to debug this... downloaded cURL from here: [http://curl.haxx.se/latest.cgi?curl=win64-ssl-sspi|http://curl.haxx.se/latest.cgi?curl=win64-ssl-sspi] The manual is here: [http://curl.haxx.se/docs/manual.html|http://curl.haxx.se/docs/manual.html]\\ curl \-u gae-client:gae-client-pw [http://140.107.149.214:8095/rest/usermanagement/1/user?username=mike.kellen|http://140.107.149.214:8095/rest/usermanagement/1/user?username=mike.kellen] returns the same 404 error that I saw in my web browser curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" [http://deflaux:8095/rest/usermanagement/1/user?username=mike.kellen|http://deflaux:8095/rest/usermanagement/1/user?username=mike.kellen] same result\! I sent this on 12/14 Dear Sirs, I have a demo version of Crowd up and running and have two questions: 1) When I try to exercise the Crowd rest api I get a "404" error from Tomcat. Specifically in cURL I send curl \-u a-client:a-client-pw \-H "Accept: application/xml" [http://<crowd|http://%3ccrowd/] server addr>:8095/rest/usermanagement/1/user?username=<a user> where <crowd server addr> is the location of my server and <a user> is a user I set up in the evaluation directory. "a-client" and "a-client-pw" are the client name and pw I set up for a generic application in the Crowd admin' utility. When I execute this http request I get back <html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><\!--H1 \{fon t-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size: 22px;\}H2 \{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525 D76;font-size:16px;\}H3 \{font-family:Tahoma,Arial,sans-serif;color:white;backgro und-color:#525D76;font-size:14px;\}BODY \{font-family:Tahoma,Arial,sans-serif;col or:black;background-color:white;\}B \{font-family:Tahoma,Arial,sans-serif;color:w hite;background-color:#525D76;\}P \{font-family:Tahoma,Arial,sans-serif;backgroun d:white;color:black;font-size:12px;\}A \{color : black;\}A.name \{color : black;\}HR \{color : #525D76;\}--></style> </head><body><h1>HTTP Status 404 - /rest/usermanag ement/1/user</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p> <p><b>message</b> <u>/rest/usermanagement/1/user</u></p><p><b>description</b> <u >The requested resource (/rest/usermanagement/1/user) is not available.</u></p>< HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.26</h3></body></html> Can you tell me where I went wrong? Is there something specific I need to do to enable the rest API beyond starting up Crowd? 2) If I use the rest api along with the SAML-based Google App's connector, will I get single sign-on across all applications? That is, if I use the rest api's SSO resource to sign-on, how does the Google App's connector "know" that I'm signed on and avoid re-authenticating? Is there something special I need to do with the token returned by the SSO resource so that the Google App's connector "finds" it? Thanks. Response: Q: Can you tell me where I went wrong? Is there something specific I need to do to enable the rest API beyond starting up Crowd? A: You don't need to do anything special to start up the REST API. I would recommend using the syntax containing the context (e.g. [http://<CROWD_SERVER>:8095/crowd/rest/usermanagement/latest/user?username=USERNAME|http://%3ccrowd_server%3e:8095/crowd/rest/usermanagement/latest/user?username=USERNAME]). Also, remember that the REST API if for applications to access Crowd. You should use the application name and password created when you set up a custom application as your credentials for the REST service ("a-client" and "a-client-pw") in your example. Q: If I use the rest api along with the SAML-based Google App's connector, will I get single sign-on across all applications? A: SSO with Google plus one or more Crowd-connected applications works. Google uses it's own mechanism, however and does not use Crowd's SSO cookie/token. Details on how SSO works in Crowd can be found here. [http://confluence.atlassian.com/display/CROWD/Overview+of+SSO|http://confluence.atlassian.com/display/CROWD/Overview+of+SSO]\\ \--------------------------------\- I'm retrying the query using context='crowd': curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" [http://deflaux:8095/crowd/rest/usermanagement/latest/user?username=mike.kellen|http://deflaux:8095/crowd/rest/usermanagement/latest/user?username=mike.kellen] This 'works', i.e. I now reach the service. But it says I'm not authorized to access it due to my IP address. This is expected. I added 140.107.176.92 as an allowed 'remote address' in the Crowd Applications tab > Remote Addresses It works\! I get back: <user name="mike.kellen" expand="attributes"> <link rel="self" href="[http://deflaux:8095/crowd/rest/usermanagement/latest/user?username=mike.kellen"/|http://deflaux:8095/crowd/rest/usermanagement/latest/user?username=mike.kellen"/]> <first-name>Mike</first-name> <last-name>Kellen</last-name> <display-name>Mike Kellen</display-name> <email>bruce.hoff@sagebase.org</email> <password> <link rel="edit" href="[http://deflaux:8095/crowd/rest/usermanagement/latest/user/password?username=mike.kellen"/|http://deflaux:8095/crowd/rest/usermanagement/latest/user/password?username=mike.kellen"/]> </password> <active>true</active> <attributes> <link rel="self" href="[http://deflaux:8095/crowd/rest/usermanagement/latest/user/attribute?username=mike.kellen"/|http://deflaux:8095/crowd/rest/usermanagement/latest/user/attribute?username=mike.kellen"/]> </attributes> </user> Let's try [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=mike.kellen&groupname=demo|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=mike.kellen&groupname=demo] <error> <reason>MEMBERSHIP_NOT_FOUND</reason> \- <message> The child entity <mike.kellen> is not a member of the parent <demo> </message> </error> created a group in Crowd called 'demo-group' and added Mike. [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=mike.kellen&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=mike.kellen&groupname=demo-group] returns <group name="demo-group"> <link rel="self" href="[http://deflaux:8095/crowd/rest/usermanagement/latest/group?groupname=demo-group"/|http://deflaux:8095/crowd/rest/usermanagement/latest/group?groupname=demo-group"/]> </group> but [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=bruce.hoff&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=bruce.hoff&groupname=demo-group] returns <error> <reason>MEMBERSHIP_NOT_FOUND</reason> \- <message> The child entity <bruce.hoff> is not a member of the parent <demo-group> </message> </error> Looks great. Now we need to exercise SSO. Per [http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+Resources|http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+Resources] we POST [http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false] with the content <?xml version="1.0" encoding="UTF-8 8 "?> <authentication-context> <username>mike.kellen</username> <password></password> <validation-factors> <validation-factor> <name>remote_address</name> <value>127.0.0.1</value> </validation-factor> </validation-factors> </authentication-context> In cURL: curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-d "%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%20%20%0D%0A%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20" [http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false] returned a 500 error curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" [http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false] <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><error><reason>INVALID_SSO_TOKEN</reason> <message>Failed to find entity of type \[com.atlassian.crowd.model.token.Token\] with identifier \[validate-password=false\]</message></error> curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-d "%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20%20" [http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false] again returns a 500 curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-d "<?xml version="1.0" encoding="UTF-8 8 "?> <authentication-context> <username>mike.kellen</username> <password></password> <validation-factors> <validation-factor> <name>remote_address</name> <value>127.0.0.1</value> </validation-factor> </validation-factors> </authentication-context> " [http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false] again returns a 500 curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \--data "%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%20%20%0D%0A%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20" [http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false] again returns a 500 I don't see where you specify the HTTP method, but this example seems to say it's with the \-X option [http://groups.google.com/group/brightkite-api/browse_thread/thread/b5d8402ed6f3ca69|http://groups.google.com/group/brightkite-api/browse_thread/thread/b5d8402ed6f3ca69] curl \-X POST \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-d "%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%20%20%0D%0A%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20" [http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false] Once I figure this out, I have to handle the cookie, perhaps as explained here: [http://www.hccp.org/java-net-cookie-how-to.html|http://www.hccp.org/java-net-cookie-how-to.html] ... then I have to get it running on GAE, which may mean rehosting the Crowd server. One note: It looks like Crowd groups don't span multiple directories. Need to check this. \\ \\ curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=bruce.hoff|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=bruce.hoff] <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?> <groups expand="group"> <group name="crowd-administrators"> <link rel="self" href="[http://deflaux:8095/crowd/rest/usermanagement/latest/group?groupname=crowd-administrators"/|http://deflaux:8095/crowd/rest/usermanagement/latest/group?groupname=crowd-administrators"/]> </group> </groups> \\ curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group] Returns <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><status><status-code>500</status-code></status>\* Closing connection #0 'groupname' is not recognized as an internal or external command, operable program or batch file. ... so it seems to be complaining about the 'groupname' param tacked on the end of the URL curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff] returns <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?> <status><status-code>500</status-code></status> \\ I keep getting the '500' error while trying to use the Crowd REST API. I finally sent another product enquiry to Atlassian with an example: I get errors using some of the Crowd REST API commands. An example is adding a user to a group. Below I give the cURL command, the result, and the stack trace in the Tomcat log file. Can anyone say whether this is a Crowd bug or what I've done wrong and, if I've done something wrong, what the correction is? Thanks\! \\ curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group] \* About to connect() to deflaux port 8095 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8095 (#0) \* Server auth using Basic with user 'gae-client' > POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8095 > Accept: application/xml > Content-Length: 66 > Content-Type: application/x-www-form-urlencoded > < HTTP/1.1 500 Internal Server Error < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Set-Cookie: JSESSIONID=6A8AF73DC2298816307074EC949B18FE; Path=/crowd < Cache-Control: no-cache, no-store, no-transform < Content-Type: application/xml < Content-Length: 102 < Date: Fri, 17 Dec 2010 23:42:41 GMT < Connection: close < <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><status><status-code>500</status-code></status>\* Closing connection #0 'groupname' is not recognized as an internal or external command, operable program or batch file. \\ from tomcat/apache-tomcat/logs/cataline.2010-12-17.log Dec 17, 2010 3:42:41 PM com.sun.jersey.server.impl.application.WebApplicationImpl onException SEVERE: Internal server error javax.ws.rs.WebApplicationException at com.sun.jersey.server.impl.uri.rules.TerminatingRule.accept(TerminatingRule.java:55) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:654) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:612) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:603) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:309) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:425) at com.atlassian.plugins.rest.module.RestDelegatingServletFilter$JerseyOsgiServletContainer.doFilter(RestDelegatingServletFilter.java:203) at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:689) at com.atlassian.plugins.rest.module.RestDelegatingServletFilter.doFilter(RestDelegatingServletFilter.java:112) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.plugins.rest.module.servlet.RestServletUtilsUpdaterFilter.doFilterInternal(RestServletUtilsUpdaterFilter.java:26) at com.atlassian.plugins.rest.module.servlet.RestServletUtilsUpdaterFilter.doFilter(RestServletUtilsUpdaterFilter.java:40) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:169) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at com.atlassian.crowd.console.filter.CrowdDelegatingFilterProxy.doFilter(CrowdDelegatingFilterProxy.java:38) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.crowd.plugin.rest.auth.BasicApplicationAuthenticationFilter.doFilter(BasicApplicationAuthenticationFilter.java:80) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198) at com.atlassian.crowd.console.filter.CrowdOpenSessionInViewFilter.doFilterInternal(CrowdOpenSessionInViewFilter.java:26) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66) at com.atlassian.crowd.plugin.rest.filter.CrowdVersionFilter.doFilter(CrowdVersionFilter.java:39) at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74) at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55) at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:636) TIME PASSES.... After some exchanges with Atlassian, it seems the main problem is that an additional header is needed in the HTTP Request, "Content-Type: application/xml". Here're the latest results: curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>mike.kellen</username> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.136</value> </validation-factor> </validation-factors> </authentication-context>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false] \* About to connect() to deflaux port 8095 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8095 (#0) \* Server auth using Basic with user 'gae-client' > POST /crowd/rest/usermanagement/1/session?validate-password=false HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8095 > Accept: application/xml > Content-Type: application/xml > Content-Length: 265 > < HTTP/1.1 400 Bad Request < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Set-Cookie: JSESSIONID=18A78725E7376FF298A2FAB502A99DD3; Path=/crowd < Content-Type: application/xml < Content-Length: 150 < Date: Wed, 22 Dec 2010 17:19:00 GMT < Connection: close < <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><error><reason>ILLEGAL_ARGUMENT</reason><message>The validated object is null</message></error>\* Closing connection #0 Updated the server IP address... curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>mike.kellen</username> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.214</value> </validation-factor> </validation-factors> </authentication-context>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false] ... but got the same result. Try empty password field... curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>mike.kellen</username> <password></password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.214</value> </validation-factor> </validation-factors> </authentication-context>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false] ... but got same result. Try filled in p/w field... curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>mike.kellen</username> <password>ssotest</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.214</value> </validation-factor> </validation-factors> </authentication-context>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false|http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false] ... same result. I sent another request to Atlassian to ask about the ILLEGAL_ARGUMENT error. Back to the add-user-to-group command. The original command was: curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group] which generated 500 error. Adding in 'content-type' header: curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group] results in 201 'created' message, but there's still the error "'groupname' is not recognized...". \* About to connect() to deflaux port 8095 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8095 (#0) \* Server auth using Basic with user 'gae-client' > POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8095 > Content-Type: application/xml > Accept: application/xml > Content-Length: 65 > < HTTP/1.1 201 Created < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Set-Cookie: JSESSIONID=27110347D1CD6EBAEBCD42CA6673DD57; Path=/crowd < Location: [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.ho|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.ho] ff&groupname=demo-group < Content-Type: application/xml < Content-Length: 0 < Date: Wed, 22 Dec 2010 17:43:27 GMT < \* Connection #0 to host deflaux left intact \* Closing connection #0 'groupname' is not recognized as an internal or external command, operable program or batch file. ... putting quotes around URL... curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "[http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group]" ... works perfectly: C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: appli cation/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "http: //deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo \-group" \* About to connect() to deflaux port 8095 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8095 (#0) \* Server auth using Basic with user 'gae-client' > POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8095 > Content-Type: application/xml > Accept: application/xml > Content-Length: 65 > < HTTP/1.1 201 Created < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Set-Cookie: JSESSIONID=CB8B969B95FFA3F6638CC7279DC7FD45; Path=/crowd < Location: [http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.ho|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.ho] ff&groupname=demo-group < Content-Type: application/xml < Content-Length: 0 < Date: Wed, 22 Dec 2010 17:51:40 GMT < \* Connection #0 to host deflaux left intact \* Closing connection #0 \\ Moreover I can log in to the Crowd admin console and see that the user is added to the group. \\ \\ h3. Crowd + HTTPS The sticking point above is that the SSO method for gettng a session token requires the user name and PASSWORD to be sent, which is not secure. To make the use of the Crowd REST API secure we want to send the request via HTTPS rather than HTTP: We enabled https on the demo crowd server by following the instructions here: [http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL|http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL]\\ Now we exercise the API: curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>mike.kellen</username> <password>ssotest</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentication-context>" \-X POST [https://deflaux:8443/crowd/rest/usermanagement/1/session?validate-password=true|https://deflaux:8443/crowd/rest/usermanagement/1/session?validate-password=true]\\ generates 403 message \\ The add-user-to-group command no longer works: curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "[https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group]" C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "[https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group]" \* About to connect() to deflaux port 8443 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8443 (#0) \* SSLv3, TLS handshake, Client hello (1): \* SSLv3, TLS handshake, Server hello (2): \* SSLv3, TLS handshake, CERT (11): \* SSLv3, TLS alert, Server hello (2): \* SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed \* Closing connection #0 curl: (60) SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed More details here: [http://curl.haxx.se/docs/sslcerts.html|http://curl.haxx.se/docs/sslcerts.html] curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the \--cacert option. If this HTTPS server uses a certificate signed by a CA represented in the bundle, the certificate verification probably failed due to a problem with the certificate (it might be expired, or the name might not match the domain name in the URL). If you'd like to turn off curl's verification of the certificate, use the \-k (or \--insecure) option. curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "[https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group]" C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "ht tps://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group" \* About to connect() to deflaux port 8443 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8443 (#0) \* SSLv3, TLS handshake, Client hello (1): \* SSLv3, TLS handshake, Server hello (2): \* SSLv3, TLS handshake, CERT (11): \* SSLv3, TLS handshake, Server key exchange (12): \* SSLv3, TLS handshake, Server finished (14): \* SSLv3, TLS handshake, Client key exchange (16): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSL connection using DHE-RSA-AES256-SHA \* Server certificate: \* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* start date: 2011-02-03 00:06:18 GMT \* expire date: 2011-05-04 00:06:18 GMT \* common name: Bruce Hoff (does not match 'deflaux') \* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* SSL certificate verify result: self signed certificate (18), continuing anyway. \* Server auth using Basic with user 'gae-client' > POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8443 > Content-Type: application/xml > Accept: application/xml > Content-Length: 65 > < HTTP/1.1 403 Forbidden < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Content-Type: text/html;charset=utf-8 < Content-Length: 1429 < Date: Thu, 03 Feb 2011 22:18:10 GMT < <html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><\!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}H2 {font-family:Tahoma,Arial,sans \-serif;color:white;background-color:#525D76;font-size:16px;}H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}BODY {font-family:Tahoma,Arial,sans-serif;col or:black;background-color:white;}B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {c olor : black;}A.name {color : black;}HR {color : #525D76;}\--></style> </head><body><h1>HTTP Status 403 - Client with address "140.107.149.249", and hostname "140.107.149.249" is fo rbidden from making requests to the application, gae-client.</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Client with address "140.107.149.249", an d hostname "140.107.149.249" is forbidden from making requests to the application, gae-client.</u></p><p><b>description</b> <u>Access to the specified resource (Client with address "140 .107.149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, gae-client.) has been forbidden.</u></p><HR size="1" noshade="noshade"><h3>Apache Tom cat/6.0.26</h3></body></html>\* Connection #0 to host deflaux left intact \* Closing connection #0 \* SSLv3, TLS alert, Client hello (1): Tried adding 140.107/16 to list of allowed addresses, but that didn't help. Tried adding 140.107.149.249, which WORKS\!\!\! C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "ht tps://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group" \* About to connect() to deflaux port 8443 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8443 (#0) \* SSLv3, TLS handshake, Client hello (1): \* SSLv3, TLS handshake, Server hello (2): \* SSLv3, TLS handshake, CERT (11): \* SSLv3, TLS handshake, Server key exchange (12): \* SSLv3, TLS handshake, Server finished (14): \* SSLv3, TLS handshake, Client key exchange (16): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSL connection using DHE-RSA-AES256-SHA \* Server certificate: \* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* start date: 2011-02-03 00:06:18 GMT \* expire date: 2011-05-04 00:06:18 GMT \* common name: Bruce Hoff (does not match 'deflaux') \* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* SSL certificate verify result: self signed certificate (18), continuing anyway. \* Server auth using Basic with user 'gae-client' > POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8443 > Content-Type: application/xml > Accept: application/xml > Content-Length: 65 > < HTTP/1.1 201 Created < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Set-Cookie: JSESSIONID=B875CF7183833A03C4E8CF6DFC0243B4; Path=/crowd; Secure < Location: [https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group] < Content-Type: application/xml < Content-Length: 0 < Date: Thu, 03 Feb 2011 22:21:41 GMT < \* Connection #0 to host deflaux left intact \* Closing connection #0 \* SSLv3, TLS alert, Client hello (1): C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "[https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=john.hill&groupname=demo-group|https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=john.hill&groupname=demo-group]" Now that I added my current IP addr into the trusted list, the 'session' command works: C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>mike .kellen</username> <password>ssotest</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authent ication-context>" \-X POST [https://deflaux:8443/crowd/rest/usermanagement/1/session?validate-password=true|https://deflaux:8443/crowd/rest/usermanagement/1/session?validate-password=true] \* About to connect() to deflaux port 8443 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8443 (#0) \* SSLv3, TLS handshake, Client hello (1): \* SSLv3, TLS handshake, Server hello (2): \* SSLv3, TLS handshake, CERT (11): \* SSLv3, TLS handshake, Server key exchange (12): \* SSLv3, TLS handshake, Server finished (14): \* SSLv3, TLS handshake, Client key exchange (16): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSL connection using DHE-RSA-AES256-SHA \* Server certificate: \* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* start date: 2011-02-03 00:06:18 GMT \* expire date: 2011-05-04 00:06:18 GMT \* common name: Bruce Hoff (does not match 'deflaux') \* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* SSL certificate verify result: self signed certificate (18), continuing anyway. \* Server auth using Basic with user 'gae-client' > POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8443 > Accept: application/xml > Content-Type: application/xml > Content-Length: 294 > < HTTP/1.1 201 Created < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Set-Cookie: JSESSIONID=F8B3FE2A89EBD0A545D16C67EA658FC5; Path=/crowd; Secure < Cache-Control: no-cache, no-store, no-transform < Location: [https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00|https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00] < Content-Type: application/xml < Content-Length: 366 < Date: Thu, 03 Feb 2011 22:33:43 GMT < <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><session expand="user"><token>JBjiPXiXVnqgEg8cpn2kWg00</token><user name="mike.kellen"><link rel="self" href="[https://deflaux:8443/crowd/rest/use|https://deflaux:8443/crowd/rest/use] rmanagement/1/user?username=mike.kellen"/></user><link rel="self" href="[https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00"/></session|https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00%22%3E%3C/a%3E%3C/session]>\* Connection #0 to host deflaux le ft intact \* Closing connection #0 \* SSLv3, TLS alert, Client hello (1): \\ Try validating an existing token: curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-X GET [https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00|https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00] C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-X GET [https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn|https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn] 2kWg00 \* About to connect() to deflaux port 8443 (#0) \* Trying 140.107.149.214... connected \* Connected to deflaux (140.107.149.214) port 8443 (#0) \* SSLv3, TLS handshake, Client hello (1): \* SSLv3, TLS handshake, Server hello (2): \* SSLv3, TLS handshake, CERT (11): \* SSLv3, TLS handshake, Server key exchange (12): \* SSLv3, TLS handshake, Server finished (14): \* SSLv3, TLS handshake, Client key exchange (16): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSLv3, TLS change cipher, Client hello (1): \* SSLv3, TLS handshake, Finished (20): \* SSL connection using DHE-RSA-AES256-SHA \* Server certificate: \* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* start date: 2011-02-03 00:06:18 GMT \* expire date: 2011-05-04 00:06:18 GMT \* common name: Bruce Hoff (does not match 'deflaux') \* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff \* SSL certificate verify result: self signed certificate (18), continuing anyway. \* Server auth using Basic with user 'gae-client' > GET /crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00 HTTP/1.1 > Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3 > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: deflaux:8443 > Accept: application/xml > Content-Type: application/xml > < HTTP/1.1 200 OK < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.0 < Set-Cookie: JSESSIONID=8E644012E156354E691D22B7C616D2CB; Path=/crowd; Secure < Content-Type: application/xml < Content-Length: 788 < Date: Thu, 03 Feb 2011 22:39:23 GMT < <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><session expand="user"><token>JBjiPXiXVnqgEg8cpn2kWg00</token><user name="mike.kellen"><link rel="self" href="[https://deflaux:8443/crowd/rest/use|https://deflaux:8443/crowd/rest/use] rmanagement/1/user?username=mike.kellen"/><first-name>Mike</first-name><last-name>Kellen</last-name><display-name>Mike Kellen</display-name><email>bruce.hoff@sagebase.org</email><password><link rel="e dit" href="[https://deflaux:8443/crowd/rest/usermanagement/1/user/password?username=mike.kellen"/></password><active>true</active><attributes><link|https://deflaux:8443/crowd/rest/usermanagement/1/user/password?username=mike.kellen"></a></password><active>true</active><attributes><link] rel="self" href="[https://deflaux:8443/crowd/rest/user|https://deflaux:8443/crowd/rest/user] management/1/user/attribute?username=mike.kellen"/></attributes></user><link rel="self" href="[https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00"/></session|https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00%22%3E%3C/a%3E%3C/session]>\* Connection #0 to host deflaux left intact \* Closing connection #0 \* SSLv3, TLS alert, Client hello (1): We also verified that we can force a token to expire. \__________________________________________________________________________________\_ Tried the original command (worked before SSO enable): curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "[http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group|http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group]" C:\Users\bhoff>curl \-u gae-client:gae-client-pw \-H "Content-Type: application/xml" \-H "Accept: application/xml" \-v \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <group name='demo-group'/>" \-X POST "http: //deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group" \* About to connect() to deflaux port 8095 (#0) \* Trying 140.107.149.214... Connection refused \* couldn't connect to host \* Closing connection #0 curl: (7) couldn't connect to host \\ So while enabling HTTPS we have also disabled HTTP. h3. \_________________________________________________________________\_ h2. Notes Q: What's the cumulative file size on the Sage SSH server? A: About 2GB, considering the files in the directory /data/incoming on sage.fhcrc.org Google Apps provides two APIs to help with authentication: SAML Single Sign-On (SSO) Service: would allow \*us\* to create and maintain users and groups outside of Google. [http://code.google.com/googleapps/domain/sso/saml_reference_implementation.html|http://code.google.com/googleapps/domain/sso/saml_reference_implementation.html] Google Apps Provisioning API: would allow us to programmatically create Google users and groups in our private domain. This would streamline adding users to Google Apps. If we used it as a total solution, then the non-google app's (e.g. Addama) would have to go to google for authentication, which violates the 'arms length' integration requirement. \\ OpenID sounds like an alternative to SAML: [http://www.google.com/support/forum/p/apps-apis/thread?tid=33a3707bd2ea7904&hl=en|http://www.google.com/support/forum/p/apps-apis/thread?tid=33a3707bd2ea7904&hl=en] In the case of OpenID, the user may have a Google Account, a Google Apps Account, or an account from any other domain that provides OpenID federated login. Integration of GAE with OpenID: [http://code.google.com/appengine/docs/java/users/overview.html|http://code.google.com/appengine/docs/java/users/overview.html]\\ h3. "What would Atlassian Do" (WWAD)? 4.1 Seraph is a very simple, pluggable J2EE web application security framework developed by Atlassian and used in our products. [http://confluence.atlassian.com/display/DEV/Single+Sign-on+Integration+with+JIRA+and+Confluence|http://confluence.atlassian.com/display/DEV/Single+Sign-on+Integration+with+JIRA+and+Confluence]\\ 4.2 Crowd is a single sign-on (SSO) application for as many users, web applications and directory servers you need --- all through a single web interface. [http://www.atlassian.com/software/crowd/|http://www.atlassian.com/software/crowd/] Crowd centralises identity management, allowing you to take users from different directories and manage them in one place. Multiple user directories can be centrally managed via Crowd's administration console. Crowd's OpenID authentication server, CrowdID, talks with websites and applications using OpenID. It expands Crowd's SSO capabilities to applications outside your organisation's firewall. \\ [http://confluence.atlassian.com/display/CROWD/Configuring+the+Google+Apps+Connector|http://confluence.atlassian.com/display/CROWD/Configuring+the+Google+Apps+Connector] To enable single sign-on in Google Apps, you will need the Premier, Education, or Partners edition of Google Apps. The Crowd Google Apps connector does not support the automatic adding of users. If a user exists in Crowd but not in Google Apps, then the user will not be able to log in to Google Apps. \\ To add an application (e.g. a GAE app like Addama registry): [http://confluence.atlassian.com/display/CROWDDEV/Application+Integration+Overview|http://confluence.atlassian.com/display/CROWDDEV/Application+Integration+Overview]\\ Licensing and hosting Crowd: \- Crowd is not hosted by Atlassian. We have to run it ourselves. It runs on Windows, Linux or Mac and uses an apache tomcat app server: [http://confluence.atlassian.com/display/CROWD/Installing+Crowd+and+CrowdID|http://confluence.atlassian.com/display/CROWD/Installing+Crowd+and+CrowdID] \- Pricing: $10 for up to 10 users then $600/$1200 for up to 100 users (academic/commercial) and $4000/$8000 for unlimited users. [http://www.atlassian.com/software/crowd/pricing.jsp|http://www.atlassian.com/software/crowd/pricing.jsp] Also the user count is the 'total number of users who can authenticate against the Crowd instance', not just the number of simultaneously active users. From [http://www.atlassian.com/software/crowd/licensing.jsp#terms|http://www.atlassian.com/software/crowd/licensing.jsp#terms] "Licensing fees are quoted per _total number of 'Crowd users'_. A Crowd user is defined as any user account that can authenticate against one or more applications." To integrate other applications with Crowd: Writing a Crowd custom application connector: [http://confluence.atlassian.com/display/CROWDDEV/Creating+a+Crowd+Client+for+your+Custom+Application|http://confluence.atlassian.com/display/CROWDDEV/Creating+a+Crowd+Client+for+your+Custom+Application] To integrate with Apache's authentication: [http://confluence.atlassian.com/display/CROWD/Integrating+Crowd+with+Apache|http://confluence.atlassian.com/display/CROWD/Integrating+Crowd+with+Apache]\\ h3. Other Single Sign-On software: Another alternative is "SSO Easy". \\ Open source alternative: [http://code.google.com/googleapps/domain/open_source_projects.html#sso|http://code.google.com/googleapps/domain/open_source_projects.html#sso]\\ Enterprise Sign-on Engine: [http://esoeproject.org/|http://esoeproject.org/]\\ h3. Other notes \- Sage SSH/SCP server authenticates using standard unix log-in. h3. Addama Authentication/Authorization Some info here [http://code.google.com/p/addama/w/list|http://code.google.com/p/addama/w/list] \- Addama authentication is via Servlet filters using GAE User Service OR a Google API-key. \- Addama handles authentication via Servlet Filters; the servlet config xml file shows what's in place. \- Addama white list: "user x can get these services, or anything under the branch." Notes on Addama Registry Filters: org.systemsbiology.addama.coresvcs.gae.filters.StaticContentFilter I don't think this has anything to do with authentication, rather it's a cache for static content. Note: You can't even get this far without being authenticated. Note: The white list (below) \*authorizes*, and doesn't apply to static content. org.systemsbiology.addama.coresvcs.gae.filters.UserServiceFilter If logged-in Google Acct OR valid API Key, then allow, else deny. org.systemsbiology.addama.coresvcs.gae.filters.WhiteListFilter If the user is an Admin or is in a 'white list' for the requested resource, then allow, else deny. org.systemsbiology.addama.coresvcs.gae.filters.DirectLinkFilter Seems to handle a specific kind of request called a 'direct link' request. (This MIGHT be a method for retrieving large files.) org.systemsbiology.addama.coresvcs.gae.filters.AdminOnlyFilter Filter out any requests NOT from an admin. Applied only for addama/memcache/\* org.systemsbiology.addama.coresvcs.gae.filters.ProxiesFilter Seems to forward certain requests (in particular, non-registry requests) to GAE's "URLFetchService". \- what does <security-constraint> in the GAE web.xml file mean? A: from [http://code.google.com/appengine/docs/java/users/overview.html] If you have pages that the user should not be able to access unless signed in, you can establish a security constraint for those pages in the deployment descriptor (the web.xml or app.yaml file). If a user accesses a URL with a security constraint and the user is not signed in, App Engine redirects the user to the sign-in page automatically (for Google Accounts or Google Apps authentication) or to the page at /_ah/login_required (for OpenID authentication), then directs the user back to the URL after signing in or registering successfully. A security constraint can also require that the user be a registered administrator for the application. This makes it easy to build administrator-only sections of the site, without having to implement a separate authorization mechanism. Nicole's "an area for testing" is a "google apps for your domain" domain [http://www.google.com/a/sagebionetworks.com|http://www.google.com/a/sagebionetworks.com] is a "test domain for Google Apps" What's the difference between a "google account" and a "google apps account"? A: the latter is newer and ultimately should subsume the former. Does Google Apps support OpenID? A: Only as an "Identity provider" (of the Google Apps ID) not as a service provider seeking authentication. [http://code.google.com/googleapps/domain/sso/openid_reference_implementation.html|http://code.google.com/googleapps/domain/sso/openid_reference_implementation.html]\\ 3 ways to authenticate GAE \- google accounts \- google-apps account (on proprietary domain associated with Google) \- OpenID ours is a google apps premier (="business"?) account \\ h3. Notes on Authorization From: [http://www.google.com/support/a/bin/answer.py?answer=61017&hl=en|http://www.google.com/support/a/bin/answer.py?answer=61017&hl=en] With Google Apps for Business and Education, administrators can use two-legged OAuth for domain-wide delegation of authority. An application that has the OAuth consumer key and secret (roughly equivalent to a role account username and password) is allowed to act as any user in the domain when accessing Google Data APIs. Unlike three-legged OAuth, users do not need to give consent on an individual basis, as this decision is made on their behalf by the administrator. \\ This is one of the admin screens in sagebionetworks.com for authorization: [https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients|https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients]\\ From: [http://www.google.com/support/forum/p/apps-apis/thread?tid=21d4f55e584e3970&hl=en|http://www.google.com/support/forum/p/apps-apis/thread?tid=21d4f55e584e3970&hl=en] Also, for access control, your SSO solution can create a cookie for the user storing his username. When a specific website page is requested, you can again use Provisioning API to determine if the username stored in cookie is part of the required Google group and authorize him. h3. A Dead End I starting implementing OAuth, but mistakenly started working on '3 legged OAuth' when in fact I wanted to implenent the much simpler '2 legged' variety. I include my notes in case we have to go back to '3 legged authorization later.' Went to [http://code.google.com/apis/accounts/docs/OAuth.html#GoogleAppsOAuth|http://code.google.com/apis/accounts/docs/OAuth.html#GoogleAppsOAuth] which says that I have to do the following sequence: 1. Get an unauthorized request token (OAuthGetRequestToken) 2. Authorize the request token (OAuthAuthorizeToken) 3. Exchange the authorized request token for an access token (OAuthGetAccessToken) 1. issue this request: [https://www.google.com/accounts/OAuthGetRequestToken|https://www.google.com/accounts/OAuthGetRequestToken] with query param's (also can go elsewhere, values are my best guesses right now): oauth_consumer_key=sandbox-sagebionetworks.appspot.com oauth_nonce= ""+Math.abs(Random.nextLong()) // (must be unsigned) oauth_signature_method=HMAC-SHA1 oauth_signature=OrB7tgg7BJtAkn9uvbo14uC9 oauth_timestamp=System.currentTimeMillis() scope=[https://apps-apis.google.com/a/feeds/group/#readonly] [https://apps-apis.google.com/a/feeds/user/#readonly|https://apps-apis.google.com/a/feeds/user/#readonly] oauth_callback= a URL our application supplies to receive a <request token, verifier> pair The response header includes (for example) oauth_token=ab3cd9j4ks73hf7g &oauth_token_secret=ZXhhbXBsZS5jb20 &oauth_callback_confirmed=true 2. Issue this request: [https://www.google.com/accounts/OAuthAuthorizeToken|https://www.google.com/accounts/OAuthAuthorizeToken] oauth_token=<the value returned above> hd=sagebionetworks.com Now our callback URL is called to receive the <reques token, verifier> pair 3. Issue this request: [https://www.google.com/accounts/OAuthGetAccessToken|https://www.google.com/accounts/OAuthGetAccessToken] oauth_consumer_key=sandbox-sagebionetworks.appspot.com oauth_token=<the request token returned from request #1> oauth_verifier=<verification code supplied in the callback> oauth_signature_method=HMAC-SHA1 oauth_signature=OrB7tgg7BJtAkn9uvbo14uC9 oauth_timestamp=System.currentTimeMillis() oauth_nonce= ""+Math.abs(Random.nextLong()) (must be unsigned) Response header contains the access code oauth_token=ab3cd9j4ks73hf7g&oauth_token_secret=ZXhhbXBsZS5jb20 I wrote a routine to do step 1, with all param's except 'scope' and 'auth_callback'. When I ran it, it returned: "400 OK" Changed the app to have valid values for the scop and callback. Tried again. Result: Result: java.io.IOException: Could not fetch URL: [https://www.google.com/accounts/OAuthGetRequestToken|https://www.google.com/accounts/OAuthGetRequestToken].... I think it's a matter of formatting the param's. I moved the params into the request header. Now the IOException goes away and I'm back to 400/OK. \\ restarted server initially there is no public DNS after a few minutes it's running, with the public DNS name: ec2-50-16-44-152.compute-1.amazonaws.com totally different from before\! Tried going to[http://ec2-50-16-44-152.compute-1.amazonaws.com:8095/crowd/console] but it failed PuTTY'd into the server: Ran 'ps', but it showed no Java process running, so I think the server is down. cd /mnt/ebs successfully, so the ebs seems to still be running. However, there are no files there and 'df' doesn't show the storage. So I think the mount was lost. The AWS console shows that the EBS volume is 'in use' and attached to the instance (i-c31579af:/dev/sdh) Back in PuTTY: sudo mount /dev/sdh /mnt/ebs Now I see the 10GB of space, but the content is gone(\!) Tried detaching and reattaching in the AWS console. Didn't work. I give up\! \__________________\_ Started over, using my sage credentials \- created "Security Group" in the AWS console with HTTP port 8095 and HTTPS port 8443. \- created a 'micro' instance "Public DNS" is: ec2-50-17-17-19.compute-1.amazonaws.com copied atlassian-crowd-2.1.1.tar.gz to /home/ec2-user gunzip atlassian-crowd-2.1.1.tar.gz sudo mv atlassian-crowd-2.1.1.tar /usr/local sudo tar \-xvf atlassian-crowd-2.1.1.tar sudo chmod \-R 777 atlassian-crowd-2.1.1 edited crowd-webapp/WEB-INF/classes/crowd-init.properties just uncommented the default so crowd.home=/var/crowd-home ./start_crowd.sh [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd] resulted in 404 error. So tomcat is running (and listening on port 8095), but Crowd isn't My guess is that it's a permissions problem. sudo chmod \-R 777 var stopped/restarted Crowd Now it works\! Went through the rest of the process of setting up the server. need to set up the application key/pw pair. Created 'Generic Application' with user/pw: platform/platform-pw For the URL I set:[http://deflaux-test.appspot.com/] Note that this is NOT https\! 'Resolve IP Address' generated 127.0.0.1, which is curious... Checked "Allow all users to authenticate." Created a user called 'demouser' with password 'demouser-pw' Created a group called 'demogroup'. Create a token C:\Users\bhoff>curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>demouser </username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica tion-context>" \-X POST [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true] \* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0) \* Trying 50.17.17.19... connected \* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0) \* Server auth using Basic with user 'platform' > POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1 > Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc= > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095 > Accept: application/xml > Content-Type: application/xml > Content-Length: 295 > < HTTP/1.1 403 Forbidden < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.1 < Content-Type: text/html;charset=utf-8 < Content-Length: 1423 < Date: Wed, 09 Feb 2011 23:37:14 GMT < <html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><\!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans \-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;col or:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {c olor : black;}A.name {color : black;}HR {color : #525D76;}\--></style> </head><body><h1>HTTP Status 403 - Client with address "140.107.149.249", and hostname "140.107.149.249" is fo rbidden from making requests to the application, platform.</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Client with address "140.107.149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, platform.</u></p><p><b>description</b> <u>Access to the specified resource (Client with address "140.107 .149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, platform.) has been forbidden.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6. 0.26</h3></body></html>\* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact \* Closing connection #0 Added the remote address. Now: C:\Users\bhoff>curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>demouser </username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica tion-context>" \-X POST [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true] \* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0) \* Trying 50.17.17.19... connected \* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0) \* Server auth using Basic with user 'platform' > POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1 > Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc= > User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3 > Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095 > Accept: application/xml > Content-Type: application/xml > Content-Length: 295 > < HTTP/1.1 201 Created < Server: Apache-Coyote/1.1 < X-Embedded-Crowd-Version: 2.1.1 < Set-Cookie: JSESSIONID=07302C98E4992295F04F0FA49BA44B05; Path=/crowd < Cache-Control: no-cache, no-store, no-transform < Location: [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMqaRBLAyQ00] < Content-Type: application/xml < Content-Length: 422 < Date: Wed, 09 Feb 2011 23:40:10 GMT < <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><session expand="user"><token>OcdmfMhMTeEUmMqaRBLAyQ00</token><user name="demouser"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazo naws.com:8095/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMq aRBLAyQ00"/></session>\* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact \* Closing connection #0 Now to get the details for the user: C:\Users\bhoff>curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-k \-X GET [http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/us] er?username=demouser <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><user name="demouser" expand="attributes"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/ user?username=demouser"/><first-name>demo</first-name><last-name>user</last-name><display-name>demo user</display-name><email>bruce.hoff@sagebase.org</email><password><link rel="edit" href="http://ec2 \-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/password?username=demouser"/></password><active>true</active><attributes><link rel="self" href="http://ec2-50-17-17-19.comput e-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/attribute?username=demouser"/></attributes></user> Now to try running from the web: changed the version in appengine-web.xml to 'repoAndAuth' then ran mvn gae:deploy from projects/trunk/services Result: 'build failure' mvn gae:run doesn't work either needed to run: 'mvn gae:unpack' Now, when I 'mvn gae:run' I can test from localhost: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://localhost:8080/repo/v1/session] {"displayName":"demo user","token":"00Cqwht6MS7mmyEiqey2Bw00"} yeah\! Next step deploy: set deployment to version 'auth' on deflaux-test.appspot.com ran: mvn gae:deploy curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://auth.latest.deflaux-test.appspot.com/repo/v1/session] got error, probably because Crowd isn't accepting messages from the app ping auth.latest.deflaux-test.appspot.com Pinging appspot.l.google.com \[74.125.127.141\] ... Added 74.125.127.141 to allowed list for 'platform' in Crowd. Still didn't work. Added more debug info, and found that this is the rejected IP: 74.125.75.17 Added the IP to the allowed list. It works\!\! curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [https://auth.latest.deflaux-test.appspot.com/repo/v1/session] Hitting GAE via https also works: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-k \-X POST [https://auth.latest.deflaux-test.appspot.com/repo/v1/session] \!\!\!\! Now to create a Crowd server that talks SSL: Created another micro instance ec2-75-101-179-108.compute-1.amazonaws.com and followed the earlier instructions to set up Crowd. BEFORE starting it up, I followed the instructions for enabling SSL:[http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL] I tweaked the Connector element in the apache-tomcat/conf/server.xml file, to add three more attributes, seen in the existing Connector element. <Connector port="8443" maxHttpHeaderSize="8192" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" enableLookups="false" disableUploadTimeout="true" acceptCount="100" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" SSLEnabled="true" keystoreFile="/home/ec2-user/.keystore" keystorePass="changeit" connectionTimeout="20000" disableUploadTimeout="true" useBodyEncodingForURI="true"/> Ran keytool \-genkey \-alias tomcat \-keyalg RSA \-sigalg MD5withRSA which creates /home/ec2-user/.keystore and I also copied the .keystore file to my desktop ('cause I think I'll need to stick it in my appengine application). Edited crowd.properties: The instructions say to edit <Crowd-Home-Directory>/crowd.properties i.e. /var/crowd-home/crowd.properties but this directory doesn't exist yet. I think it's created upon start up. An educated guess is to edit it here: /usr/local/atlassian-crowd-2.1.1/etc/crowd.properties with the expectation that it will ultimately be moved to /var/crowd-home. But looking at this file I see only " (CROWD)", not the protocol specific URLs. Made apache-tomcat/bin/setenv.sh look like JAVA_OPTS="-Xms128m \-Xmx512m \-XX:MaxPermSize=256m \-Dfile.encoding=UTF-8 $JAVA_OPTS \-Djavax.net.ssl.keyStore=/home/ec2-user/.keystore -Djavax.net.ssl.keyStorePassword=changeit \-Djavax.net.ssl.trustStore=/home/ec2-user/.keystore \-Djavax.net.ssl.trustStorePassword=changeit" export JAVA_OPTS Ran /user/local/atlassian-crowd-2.1.1/start_crowd.sh but I don't see /var/crowd-home ps doesn't show any Java process running. One problem is that I entered disableUploadTimeout="true" twice in the connector. After removing the duplicate and running 'start_crowd.sh' again, ps shows a Java process running. Still hasn't made /var/crowd-home but log files don't show any errors. Try going to [https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/console] I had to go through the 'untrusted certificate' warning in Firefox, but it works. Went through the rest of the server set up. Note: The settings include a max number of days 'til the user has to change her password. For now I'm not using this. Added demouser, demouser-pw Added application with user/pw: platform/platform-pw Now let's try to generate a session from curl: curl \-u platform:platform-pw \-H "Accept: application/xml" \-H "Content-Type: application/xml" \-v \-k \-d "<?xml version='1.0' encoding='UTF-8 8 '?> <authentication-context> <username>demouser</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentication-context>" \-X POST [https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session?validate-password=true] Got the error saying I have to add 140.107.149.249 After adding this address to the 'allowed' list, it works. The response is: <?xml version="1.0" encoding="UTF-8 8 " standalone="yes"?><session expand="user"><token>2ffROYdzaeThREyIPbV7Wg00</token><user name="demouser"><link rel="self" href="https://ec2-75-101-179-108.compute-1.a mazonaws.com:8443/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session/2ffROYd zaeThREyIPbV7Wg00"/></session> Tried running Java app. It fails because I don't have the necessary certificate in Java's keystore. The documentation seems to indicate that this isn't a problem for appengine's 'fetch' service. Now try to hit the service, running locally: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"user\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://localhost:8080/repo/v1/session] ooops it's changed: curl \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"userId\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [http://localhost:8080/repo/v1/session] Didn't work: fetch service runs but prints message: java.io.IOException: Could not fetch URL: [https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/latest/session?validate-password=true] Could it be that my IP address has changed? It is 140.107.177.53 Yes, that's new\! I added it and reran the request: ... with the same result. Just for fun, try deploying to a new app version called 'ssl'. The test is now: curl \-k \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"userId\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST [https://ssl.latest.deflaux-test.appspot.com/repo/v1/session] fails :^( Looking at the stack trace in the GAE console, it's rejecting the IP address 74.125.66.80. Added the address in the Crowd application console and reran: fails :^( hey, the address changed again, this time to: 74.125.75.1 Added this address .... it worked\!\! C:\Users\bhoff\eclipse projects\trunk\services\repository>curl \-k \-H "Content-Type:application/json" \-H "Accept:application/json" \-d "{\"userId\":\"demouser\", \"pw\":\"demouser-pw\"}" \-X POST https ://ssl.latest.deflaux-test.appspot.com/repo/v1/session {"displayName":"Demo User","token":"ih9byQiH0m80PbxKY10Byw00"} |
Page Comparison
General
Content
Integrations
App links