Authentication & Authorization for Sprint 2
This section reflects a revision of the original problem and it's investigation, captured below for historical purposes.
Goal:
Provide authentication and authorization for users of the Platform, who may come in (1) through the GWT web UI or (2) from the R command line.
Approach:
Use two services, an 'authentication' service that verifies user credentials, and a repository service that provides access to Platform resources. Both systems have the same user-ids, but only the authentication service knows about passwords.
Authentication for the Platform
- Set-up Crowd running HTTPS.
- Add users to a directory linked to Crowd, or to the Crowd database itself.
- Add an authentication service, accessed via HTTPS. This service ‘talks’ to Crowd via its REST API over HTTP. The methods are described here:
http://sagebionetworks.jira.com/wiki/display/PLFM/Authentication+and+Authorization+API
The only publicly available methods are 'log-in', 'log-out' and 'reset-password'.
- Add a filter that applies to all Repository Services calls, which validates the token, passed in the header of the request
Sequence of events:
Authorization for the Platform
High Level Architecture
The service for creating and updating users will both talk to Crowd (via HTTPS) and to the persistence layer via its DAO interface.
Persistence Model for Users and Groups
Object Access Rules
DAOFactory methods for getting DAOs now take userId as parameter, but allow null to indicate anonymous access.
When an object is first created, only the creator has access, unless the object is created anonymously, in which case the object is public.
As a default, the ability to create users and groups is public. This is necessary to 'bootstrap' the system, after which public rights may be removed.
The default access types are READ, CHANGE, and SHARE. However the types are open ended, allowing such types useful to external components, such as DOWNLOAD, to indicate that the access to metadata (READ) is separate from the access to externally referenced data (DOWNLOAD).
DAO Methods
DAOFactory:
UserDAO getUserDAO(String userId)
UserGroupDAO getUserGroupDAO(String userId)
UserDAO
standard CRUD
Collection<UserGroup> whoHasAccess(reourceId, accessType)
boolean hasAccess(resourceId, accessType)
UserGroupDAO
standard CRUD
add/remove/get all Users
add/remove/get resources
TO DO: add/remove creation rights
Notes:
- CRUD methods throw UnauthorizedException
- query methods don't throw this exception but may limit what is returned based on authorization.
Unfinished (as of 2/18/2011)
Test authentication filter in repo' service and passing userId to DAOs.
Get reset password email working (Brian Holt is currently investigating Crowd's email problem).
Repo web services for User CRUD should also call Crowd's API AND/OR we should add a process to 'mirror' Crowd users & groups to the persistence layer.
Add userId to DAO retrieval for existing repo web services.
Authorization changes still needed in these classes:
GAEJDORevisableDAOImpl
GAEJDOAnnotationDAOImpl
GAEJDORevisableDAOImpl
GAEJDOAnnotationDAOImpl
GAEJDOInputDataLayerDAOImpl
GAEJDORevisableAnnotatableDAOImpl
GAEJDORevisableAnnotationDAOImpl
GAEJDOUserGroupDAOImpl
Crowd on Amazon Elastic Compute Cloud
AWS notes:
- created an 'm1.small'
- created an EBS volume in the same zone as the server, 'us-east-1c'
Followed instructions to attach volume:http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?using-ebs.html
My server is in
made new volume, in the right 'zone', vol-e47e198c
In 'attach volume' dialog, specified '/dev/sdh'. Nicole's advice is:
in the AWS console choose a mount point of /dev/sdh
log onto the box and run
sudo mount /dev/xvdh /mnt/myNewExternalDrive
(The dialog says "Linux Devices: /dev/sdf through /dev/sdp")
In PuTTY (following http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?using-ebs.html)
yes | sudo /sbin/mkfs -t ext3 /dev/sdh
sudo mkdir /mnt/ebs
sudo mount /dev/sdh /mnt/ebs
downloaded
atlassian-crowd-2.1.1.tar.gz
i.e. the Unix distribution and then uploaded it to
/home/ec2-user/
It takes many minutes to upload.
(I first tried uploading directly to /mnt/ebs but it didn't work.)
sudo cp'd the file to /mnt/ebs
Nicole's advice:
sudo chown -R ec2-user:ec2-user ebs/
gzip -d atlassian-crowd-2.1.1.tar.gz
tar -xvf atlassian-crowd-2.1.1.tar
so the file structure is now:
/mnt/ebs/atlassian-crowd-2.1.1
set crowd.home in the crowd-init.properties file to
/mnt/ebs/crowd-home
** will use the evaluation database for now **
cd'd to
/mnt/ebs/atlassian-crowd-2.1.1
and typed
./start_crowd.sh
it seems to have started up
Also note: I haven't enabled HTTPS yet!
Need to go through these instructions:
http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL
Now go to
http://ec2-50-17-14-80.compute-1.amazonaws.com:8095
It works!
I then went back to the EC2 console and STOPPED the server
i-c31579af
Note: I did not run the Crowd shut down script first!
Server ID is
BCT7-MZ9X-K8I8-PM3U
generated a demo license and installed
for database, chosen 'embedded' (i.e. demo version)
For 'password encryption' I chose 'atlassian security'.
For email server I used smtp.gmail.com with bruce.hoff and my pw. I left SSL off (which may be wrong).
I bypassed setting up the OpenID integration.
DONE!
http://ec2-50-17-14-80.compute-1.amazonaws.com:8095/crowd/console/login.action
Next steps:
1) Restart and see if I can log in. (May have new server name; may have to restart tomcat.)
2) Enable HTTPS (SSL).
Restarted server
initially there is no public DNS
after a few minutes it's running, with the public DNS name:
ec2-50-16-44-152.compute-1.amazonaws.com
totally different from before!
Tried going tohttp://ec2-50-16-44-152.compute-1.amazonaws.com:8095/crowd/console
but it failed
PuTTY'd into the server:
Ran 'ps', but it showed no Java process running, so I think the server is down.
cd /mnt/ebs successfully, so the ebs seems to still be running.
However, there are no files there and 'df' doesn't show the storage.
So I think the mount was lost.
The AWS console shows that the EBS volume is 'in use' and attached to the instance (i-c31579af:/dev/sdh)
Back in PuTTY:
sudo mount /dev/sdh /mnt/ebs
Now I see the 10GB of space, but the content is gone(!)
Tried detaching and reattaching in the AWS console.
Didn't work. I give up!
Started over, using my sage credentials
- created "Security Group" in the AWS console
with HTTP port 8095 and HTTPS port 8443.
- created a 'micro' instance
"Public DNS" is:
ec2-50-17-17-19.compute-1.amazonaws.com
copied atlassian-crowd-2.1.1.tar.gz to /home/ec2-user
gunzip atlassian-crowd-2.1.1.tar.gz
sudo mv atlassian-crowd-2.1.1.tar /usr/local
sudo tar -xvf atlassian-crowd-2.1.1.tar
sudo chmod -R 777 atlassian-crowd-2.1.1
edited crowd-webapp/WEB-INF/classes/crowd-init.properties
just uncommented the default so
crowd.home=/var/crowd-home
./start_crowd.sh
http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd
resulted in 404 error. So tomcat is running (and listening on port 8095), but Crowd isn't
My guess is that it's a permissions problem.
sudo chmod -R 777 var
stopped/restarted Crowd
Now it works!
Went through the rest of the process of setting up the server.
need to set up the application key/pw pair.
Created 'Generic Application' with user/pw: platform/platform-pw
For the URL I set:http://deflaux-test.appspot.com/
Note that this is NOT https!
'Resolve IP Address' generated 127.0.0.1, which is curious...
Checked "Allow all users to authenticate."
Created a user called 'demouser' with password 'demouser-pw'
Created a group called 'demogroup'.
Create a token
C:\Users\bhoff>curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>demouser
</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica
tion-context>" -X POST http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true
* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0)
* Trying 50.17.17.19... connected
* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0)
* Server auth using Basic with user 'platform'
> POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1
> Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc=
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095
> Accept: application/xml
> Content-Type: application/xml
> Content-Length: 295
>
< HTTP/1.1 403 Forbidden
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.1
< Content-Type: text/html;charset=utf-8
< Content-Length: 1423
< Date: Wed, 09 Feb 2011 23:37:14 GMT
<
<html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><!--H1
H2
H3
BODY
B
P
A
A.name
HR
--></style> </head><body><h1>HTTP Status 403 - Client with address "140.107.149.249", and hostname "140.107.149.249" is fo
rbidden from making requests to the application, platform.</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Client with address "140.107.149.249", and
hostname "140.107.149.249" is forbidden from making requests to the application, platform.</u></p><p><b>description</b> <u>Access to the specified resource (Client with address "140.107
.149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, platform.) has been forbidden.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.
0.26</h3></body></html>* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact
* Closing connection #0
Added the remote address. Now:
C:\Users\bhoff>curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>demouser
</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica
tion-context>" -X POST http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true
* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0)
* Trying 50.17.17.19... connected
* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0)
* Server auth using Basic with user 'platform'
> POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1
> Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc=
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095
> Accept: application/xml
> Content-Type: application/xml
> Content-Length: 295
>
< HTTP/1.1 201 Created
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.1
< Set-Cookie: JSESSIONID=07302C98E4992295F04F0FA49BA44B05; Path=/crowd
< Cache-Control: no-cache, no-store, no-transform
< Location: http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMqaRBLAyQ00
< Content-Type: application/xml
< Content-Length: 422
< Date: Wed, 09 Feb 2011 23:40:10 GMT
<
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><session expand="user"><token>OcdmfMhMTeEUmMqaRBLAyQ00</token><user name="demouser"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazo
naws.com:8095/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMq
aRBLAyQ00"/></session>* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact
* Closing connection #0
Now to get the details for the user:
C:\Users\bhoff>curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -k -X GET http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/us
er?username=demouser
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><user name="demouser" expand="attributes"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/
user?username=demouser"/><first-name>demo</first-name><last-name>user</last-name><display-name>demo user</display-name><email>bruce.hoff@sagebase.org</email><password><link rel="edit" href="http://ec2
-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/password?username=demouser"/></password><active>true</active><attributes><link rel="self" href="http://ec2-50-17-17-19.comput
e-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/attribute?username=demouser"/></attributes></user>
Now to try running from the web:
changed the version in appengine-web.xml to 'repoAndAuth' then ran
mvn gae:deploy
from projects/trunk/services
Result: 'build failure'
mvn gae:run doesn't work either
needed to run: 'mvn gae:unpack'
Now, when I 'mvn gae:run' I can test from localhost:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://localhost:8080/repo/v1/session
yeah! Next step deploy:
set deployment to version 'auth' on deflaux-test.appspot.com
ran: mvn gae:deploy
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://auth.latest.deflaux-test.appspot.com/repo/v1/session
got error, probably because Crowd isn't accepting messages from the app
ping auth.latest.deflaux-test.appspot.com
Pinging appspot.l.google.com [74.125.127.141] ...
Added 74.125.127.141 to allowed list for 'platform' in Crowd.
Still didn't work. Added more debug info, and found that this is the rejected IP: 74.125.75.17
Added the IP to the allowed list. It works!!
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST https://auth.latest.deflaux-test.appspot.com/repo/v1/session
Hitting GAE via https also works:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -k -X POST https://auth.latest.deflaux-test.appspot.com/repo/v1/session
!!!!
Now to create a Crowd server that talks SSL:
Created another micro instance
ec2-75-101-179-108.compute-1.amazonaws.com
and followed the earlier instructions to set up Crowd.
BEFORE starting it up, I followed the instructions for enabling SSL:http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL
I tweaked the Connector element in the apache-tomcat/conf/server.xml file, to add three more attributes,
seen in the existing Connector element.
<Connector port="8443" maxHttpHeaderSize="8192" maxThreads="150"
minSpareThreads="25" maxSpareThreads="75" enableLookups="false"
disableUploadTimeout="true" acceptCount="100" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
SSLEnabled="true"
keystoreFile="/home/ec2-user/.keystore" keystorePass="changeit"
connectionTimeout="20000" disableUploadTimeout="true" useBodyEncodingForURI="true"/>
Ran
keytool -genkey -alias tomcat -keyalg RSA -sigalg MD5withRSA
which creates /home/ec2-user/.keystore
and I also copied the .keystore file to my desktop ('cause I think I'll need to stick it in my appengine application).
Edited crowd.properties: The instructions say to edit
<Crowd-Home-Directory>/crowd.properties
i.e.
/var/crowd-home/crowd.properties
but this directory doesn't exist yet. I think it's created upon start up.
An educated guess is to edit it here:
/usr/local/atlassian-crowd-2.1.1/etc/crowd.properties
with the expectation that it will ultimately be moved to /var/crowd-home.
But looking at this file I see only " (CROWD)", not the protocol specific URLs.
Made apache-tomcat/bin/setenv.sh
look like
JAVA_OPTS="-Xms128m -Xmx512m -XX:MaxPermSize=256m -Dfile.encoding=UTF-8 $JAVA_OPTS -Djavax.net.ssl.keyStore=/home/ec2-user/.keystore
-Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.trustStore=/home/ec2-user/.keystore -Djavax.net.ssl.trustStorePassword=changeit"
export JAVA_OPTS
Ran /user/local/atlassian-crowd-2.1.1/start_crowd.sh
but I don't see /var/crowd-home
ps doesn't show any Java process running.
One problem is that I entered
disableUploadTimeout="true"
twice in the connector.
After removing the duplicate and running 'start_crowd.sh' again, ps shows a Java process running.
Still hasn't made /var/crowd-home
but log files don't show any errors.
Try going to https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/console
I had to go through the 'untrusted certificate' warning in Firefox, but it works.
Went through the rest of the server set up.
Note: The settings include a max number of days 'til the user has to change her password.
For now I'm not using this.
Added demouser, demouser-pw
Added application with user/pw: platform/platform-pw
Now let's try to generate a session from curl:
curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>demouser</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentication-context>" -X POST https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session?validate-password=true
Got the error saying I have to add 140.107.149.249
After adding this address to the 'allowed' list, it works. The response is:
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><session expand="user"><token>2ffROYdzaeThREyIPbV7Wg00</token><user name="demouser"><link rel="self" href="https://ec2-75-101-179-108.compute-1.a
mazonaws.com:8443/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session/2ffROYd
zaeThREyIPbV7Wg00"/></session>
Tried running Java app. It fails because I don't have the necessary certificate in Java's keystore.
The documentation seems to indicate that this isn't a problem for appengine's 'fetch' service.
Now try to hit the service, running locally:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://localhost:8080/repo/v1/session
ooops it's changed:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://localhost:8080/repo/v1/session
Didn't work: fetch service runs but prints message:
java.io.IOException: Could not fetch URL: https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/latest/session?validate-password=true
Could it be that my IP address has changed?
It is 140.107.177.53
Yes, that's new! I added it and reran the request:
... with the same result.
Just for fun, try deploying to a new app version called 'ssl'. The test is now:
curl -k -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST https://ssl.latest.deflaux-test.appspot.com/repo/v1/session
fails :^(
Looking at the stack trace in the GAE console, it's rejecting the IP address 74.125.66.80. Added the address in the Crowd application console and reran:
fails :^(
hey, the address changed again, this time to: 74.125.75.1
Added this address .... it worked!!
C:\Users\bhoff\eclipse projects\trunk\services\repository>curl -k -H "Content-Type:application/json" -H "Accept:application/json" -d "{\"userId\":\"demouser\", \"pw\":\"demouser-pw\"}" -X POST https
://ssl.latest.deflaux-test.appspot.com/repo/v1/session
{"displayName":"Demo User","token":"ih9byQiH0m80PbxKY10Byw00"}
Following some advice found on-line, got the desktop application to work as well.
http://www.exampledepot.com/egs/javax.net.ssl/trustall.html
http://stackoverflow.com/questions/2186543/java-secure-webservice
Original Problem Formulation:
Goal:
Provide authentication / authorization for users of the Sage Platform.
Authentication: Verify user's identity.
Authorization: Allow a user, or an application invoked by user, to access data in the Platform.
The Platform comprises:
- Addama registry (running on Google App Engine, or "GAE")
- Addama feed service (on GAE)
- Addama file repository service (running on Amazon Elastic Compute Cloud, or "EC2")
- Addama Java Content Repository (JCR) service (on EC2)
- UI html files (on EC2)
- a Google Group
- shared Google Docs
- file repository, hosted at Sage, accessible via Secure CoPy (SCP)
Requirements and Design Constraints:
- Single sign-on. Once a user has signed on to the platform, they don't have to sign into any of the components.
- One-stop user administration: Adding or removing a user in one place will apply to all components.
- "Security at all layers": No component can be part of the platform unless it adheres to (one of) our authentication mechanism(s). (Note, we could implement several mechanisms "under the hood", if the systems we are integrating require it.)
- Platform will have 'arms length' integration with Google Apps, Groups. (I.e. the rest of the system (Addama, Sage file repository) must work if Google tools are omitted.)
- We want to have full control over the UI (hence a custom approach using GWT instead of GoogleSites) but link to the relevant GoogleDocs and GoogleGroups and use the GoogleDocs UI and GoogleGroups UI when folks are interacting with those resources.
Use Cases
- create a google doc, share with the group.
- send a message to the group
- create an analysis, share with the group
- add a user to the platform
- add all users from a collaborating lab to the platform
- create user credentials for a new user, providing access to the platform
- give a user or a group of users access to an addama service
Analysis
There are just four components that need to perform authentication. (The others delegate authentication to the registry.) They are listed here, along with authentication options:
Addama registry GAE application (Google account, Google Apps account, OpenID federated authentication)
Google Apps (Google Apps account, SAML delegated authentication)
Google Group (Google account, Google Apps account)
Sage SSH server (standard unix login)
If the SSH server were eliminated (by migrating the hosted files to an Addama repository service) then a common denominator *might* be Google App account authentication, which in turn might be delegated to an external identity provider.
Design
This is a possible design approach, contingent upon the answers to certain open questions (listed below):
- Restrict the Addama registry application to be hosted on a proprietary domain, configured to authenticate via "Google Apps for your domain".
- Configure Google Apps on our domain to delegate authentication via SAML.
- Configure Google Groups on our domain to delegate authentication via SAML.
- Migrate local file repository to Addama service.
- Employ Atlassian Crowd as the administration console for user authentication.
Open Questions
- Are Atlassians Crowd pricing, license models, and hosting options acceptable for our purposes? Do they prohibit integrating with NextBio?
(Note: Atlassian doesn't host Crowd, rather we download and host it ourselves. It's an Apache Tomcat application, with a variety of choices for databases.)
- What other SAML or OpenID identity provider (ip) tools (provding UIs and/or aggregating other IdP's) are there?
- Can Google Apps and Google Groups use OpenID (instead of SAML) for authentication?
NO. They will authenticate 3rd party app's via OpenID but they do not delegate authentication via OpenID.
-Do we want to use google app's to see content we host elsewhere, or will google app's be the only place that doc's are stored in this 'sprint'?
- Can "Google Group" membership be managed by an external authentication mechanism? YES, it's part of the Google Apps delegation, see below.
- if we are doing "arm's length" integration with google app's, then what other providers should we plan for?
- do we need 'audit logs', e.g. to show when users were added/removed and by whom?
Pivotal assumptions/questions (i.e. that can substantially change the design)
- Does a google doc created outside of the domain need to be able to be shared with a group in the domain?
- Do new users need to be given immediate access to the platform (without an admin in the loop)?
Experiment to address key questions
1. Authenticate Google Apps, Groups using SAML and Crowd
- Set-up Crowd trial edition (on local box or AWS)
- Change Google Apps demo domain to authenticate against Crowd
- Add user to Crowd
-Try to access Google Apps via this user (e.g. make a document)
- try to access Google Groups via this user
2. Authenticate GAE app using SAML and Crowd
- Change/deploy GAE app, authenticating via Google Apps
- Try to log into to GAE app via this user
(If not, can GAE OpenID option work with Crowd or can bypass UserService to use some sort of OpenID connector to reach Crowd?)
3. Authorize using SAML, Crowd
- Define a group in Crowd
- Add a user to a group in Crowd
- Add a user to a group in Google Apps
- See if access to services can be selected based on such group membership.
4. Replace Crowd with Open Source Identity Provider
Repeat 1-3 above.
Experiment execution
Set Up Crowd
- Added 140.107.149.214 deflaux to C:/windows/system32/drivers/etc/hosts on my la top. I can now PuTTY/SSH into 'deflaux' which is a Linux box in Nicole's office.
- Followed http://confluence.atlassian.com/display/CROWD/Installing+Crowd+and+CrowdID
- downloaded zip file
- downloaded and installed WinSCP; connected to deflaux:22 using SCP protocol.
- unzipped zip file and copied contents to /usr/local/tomcat on deflaux
- per the instructions, created the directory /var/crowd-home and edited .../crowd-webapp/WEB-INF/classes/crowd-init.properties accordingly.
- ran sudo ./start_crowd.sh
but got
"The BASEDIR environment variable is not defined correctly. This environment variable is needed to run this program"
- Googled around for a solution. Found
- Did a bunch of trial-and-error and ended up trying
chmod -R 777 *
which seemed to do the trick: Instead of getting an error message I got:
Using CATALINA_BASE: /usr/local/tomcat/apache-tomcat
Using CATALINA_HOME: /usr/local/tomcat/apache-tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/apache-tomcat/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/local/tomcat/apache-tomcat/bin/bootstrap.jar
- Instructions say to go to http://localhost:8095/crowd. I tried going to http://deflaux:8095/crowd and http://140.107.149.214:8095/crowd, but neither worked. Used 'sudo ps' and 'sudo lsof -i :8095' to show that the server is indeed running.
- Nicole poked hole in the firewall on the box. Now I can go to http://deflaux:8095/ and see the web page. Woo hoo.
- Per the instructions at
I created the file /usr/local/tomcat/crowd.init.d, then from /etc/init.d
sudo ln /usr/local/tomcat/crowd.init.d crowd
Ran the web-based set up wizard: Got a 30 day license key and chose to use the 'embedded' database.
Change Google Apps demo domain to authenticate against Crowd
Following:
http://confluence.atlassian.com/display/CROWD/Configuring+the+Google+Apps+Connector
Note, this says "you will need the Premier, Education, or Partners edition of Google Apps." so I may not be able to use 42stories.com. I'll see how far I get before I'm stuck.
- Since we're using JDK 1.6, I followed the instructions which said to put the following two jars in <Crowd-Install>/crowd-webapp/WEB-INF/lib (<Crowd-Install>=/usr/local/tomcat)
1-) xml-security-1.4.2.jar
2-) commons-logging-1.1.1.jar
- At step 1.5 "...select one or more user directories..." I picked the single one listed, "Evaluation." I believe this is the default user directory I set up during installation.
Set "Allow all to authenticate" -> True.
Under "Permissions" allow Google Apps to add/modify/remove groups and users, but I'm not sure if Google Apps can actually do this. (Perhaps it can, rhrough the Provisioning API!)
Step 2: Generate new keys. Afterwards the Configuration tab displayed:
Sign-in Page URL:
http://deflaux:8095/crowd/console/plugin/secure/saml/samlauth.action Sign-out Page URL: http://deflaux:8095/crowd/console/logoff.action Change Password URL: http://deflaux:8095/crowd/console/user/viewchangepassword.action DSA Key-pair Location: /var/crowd-home/plugin-data/crowd-saml-plugin
- Step 3. Configuring Google Apps to Recognise Crowd
Went to 42stories google app's console: https://www.google.com/a/cpanel/42stories.com/Dashboard
There is no "'single sign-on (SSO)' link."
Switched to sagebionetworks.com, which DOES have a premier version of Google Apps. Followed Atlassian instructions to set up SSO.
Note: To Disable: Go to https://www.google.com/a/cpanel/sagebionetworks.com/SetupSSO, unclick "Enable Single Sign-on", then Save Changes.
Step 4, trying it out:
I created a user called 'ssotest' having the same password. Performed the 'Authentication Test' which was successful.
Now for a true test, connecting to Google Apps on bionetworks.com using 'ssotest':
Went to http://sites.google.com/a/sagebionetworks.com
Click on 'sign in to Sage Bionetworks'
Entered ssotest / ssotest
got "Google Apps - Invalid Email" error
I *can* log in to bruce.hoff. This is because
sagebionetworks already has a bruce.hoff
Added a 'mike.kellen' pw: ssotest to Crowd
It works!
Added 'nicole.deflaux', pw: drizzle to Crowd
It works!
Conclusion: GoogleApps delegates password management, but not user management!!
Went to groups.google.com/a/sagebionetworks.com
It works! I.e. google groups delegates authentication too.
Big open question: If Crowd aggregates two directories, both having a user called john_smith, then whose credentials are used to log in to Google Apps?
Tried running Nicole's demo. Result: Was prompted for regular (non-Crowd) credentials. So this demo doesn't automatically delegate when google apps does.
This might be due to how the application was deployed. The application is associated with the sagebase.org domain, i.e. it is visible at:
https://appengine.google.com/a/sagebase.org
at the authentication choice is "Google Accounts API: The Google Accounts API includes all Gmail Accounts, but does not include accounts on Google Apps domains."
Info on how to deploy to a domain is here:
http://code.google.com/appengine/articles/auth.html
Create a Google App Engine application using Google Apps accounts to log-in
Installed GAE plug-in for Eclipse. It includes SDK v. 1.3.8.
Created an app 'sandbox-sagebionetworks.appspot.com' set to authenticate against users in the sagebionetworks.com domain. Verified that the default app runs on the web.
Added a <security-constraint> to the web.xml and redeployed. Result: I get a "500 Internal Server Error" error when I click on the servlet link. In the appengine control panel error log I see the message:
"Authentication for the Google Apps domain sagebionetworks.com can only be performed when requests are served from a subdomain of that domain or it has been approved through the Google Apps Control Panel."
I logged in using my sagebionetworks.com credentials, but got the same error again.
The problem is that I have not told GAE to delegate to Google Apps.
Went to appengine.google.com under the sandbox-sagebionetworks app
clicked 'Application Settings' then went to 'Add Domain'
entered 'sagebionetworks.com
got the message:
Your users can access sandbox-sagebionetworks at:
https://sandbox-sagebionetworks.appspot.com
Now it works! I can go to
https://sandbox-sagebionetworks.appspot.com/, click on the link, and get "Hello, world"
I am signed in as bruce.hoff@sagebionetworks.com.
I click 'Sign Out' from Google Apps and try the app url again.
Unexpectedly I CAN get to 'hello, world' (no authentication)
I close all windows
I try to go to google.com/a/sagebionetworks.com and am prompted for a log-in.
Now I go to the app, click on the "Sandbox" link and am prompted for a log-in. (Yea!)
Logged out of Google Apps
https://www.google.com/a/sagebionetworks.com/
then returned to the app and was prompted for a log-in,
so authentication seems to be working.
Now to delegate authentication to Crowd:
Logged in to the control panel for the sagebionetworks.com domain and went to 'advanced tools'
https://www.google.com/a/cpanel/sagebionetworks.com/Advanced#Advanced/subtab=0
Went to 'set up single sign-on (SSO)"
https://www.google.com/a/cpanel/sagebionetworks.com/SetupSSO
and clicked "Enable Single Sign-on" then "Save changes"
Logged out.
Went to
sites.google.com/a/sagebionetworks.com
and got the modified Google log in reflecting that SSO is activated.
Clicked "Sign in to Sage Bionetworks"
and got the Crowd log in screen.
Did NOT log in but rather went to
https://sandbox-sagebionetworks.appspot.com
Clicked on 'Sandbox'
and went to the Crowd log-in screen!! (Success!!)
Entered user: nicole.deflaux, p/w: drizzle (avoiding my own, administrative credentials)
Successfully ran the "Hello world" servlet.
Summary:
Google App Engine (GAE) can be configured to delegate authentication to Google Apps (on our domain),
which can in turn delegate authentication to an external SAML-based Identity
Provider. Moreover, the authentication requirement for the GAE services can be completely
managed in the web.xml file using <security-constraint> tags.
In principle we can create a collaborative platform including Google Apps, Google Groups,
and Google App Engine authenticating web services (e.g. Addama) in which users experience
single-sign on using their native credentials and externally managed passwords.
Authorize using SAML, Crowd
- Define a group in Crowd: Crowd defines the 'crowd-administrators' group.
- Add a user to a group in Crowd: the user 'bruce.hoff' is in the 'crowd-administrators' group.
- Add a user to a group in Google Apps: there is a group 'demo' having several members.
- See if access to services can be selected based on such group membership.
Note: Crowd does not see the google group 'demo'; Google Apps does not see the Crowd group 'crowd-administrators.
This suggests that the group definition component of authorization is not delegated by Google Apps.
The continuation of this experiment is to experiment with control of access via the Google Apps group.
Assuming that Google Apps groups 'authorize' access to Google Apps (doc's, sites, group threads, etc.),
then the two open questions are:
1) Can a web app (e.g. the Addama registry) perform authorization against the Google Apps groups;
2) Can group membership be controlled programmatically. (This is a minor need, since the Google Apps control panel allows administration of group membership.)
Connect GAE to Google Apps for group-based authorization, using 2-legged OAuth
To give the GAE app access to users and groups on the sagebionetworks.com domain, I went to
https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients
and entered
Client Name: sandbox-sagebionetworks.appspot.com
One or more API Scopes: https://apps-apis.google.com/a/feeds/group/#readonly, https://apps-apis.google.com/a/feeds/user/#readonly
the scopes came from:
http://www.google.com/support/a/bin/answer.py?answer=162106
Result: Got the error:
This client name has not been registered with Google yet.
Followed the instructions here:
http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html
1. Went to here to register the app:
https://www.google.com/accounts/ManageDomains
1.1 Entered https://sandbox-sagebionetworks.appspot.com
1.2 We to the "verify ownership" page and chose the "Upload an HTML file to your server" option
downloaded the generated html file into the 'war' folder of my GAE app
completed the 'verify ownership' step
1.3 Targe URL path prefix: https://sandbox-sagebionetworks.appspot.com/authsub
I did not complete the section 'upload new X.509 cert' (more at http://code.google.com/apis/gdata/docs/auth/authsub.html)
but clicked 'save'. Result:
OAuth Consumer Key: sandbox-sagebionetworks.appspot.com
OAuth Consumer Secret: OrB7tgg7BJtAkn9uvbo14uC9
To continue the process I need to return here: https://www.google.com/accounts/UpdateDomain
following these instructions: http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html
Next:
Need to figure out what to put at the 'authsub' URL
Went back to https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients
and clicked 'authorize' again.
This time it worked!!
Time to write the web app.
On the subject of 2-legged OAuth, from
http://code.google.com/apis/accounts/docs/OAuth.html#WorkingOauth
"This differs from the normal authorization flow, also known as 3-legged OAuth, in
that no access token is required. All applications using 2-legged OAuth must be
registered with Google."
There's a code example here:
http://code.google.com/apis/gdata/docs/auth/oauth.html#2LeggedOAuth
another example:
http://code.google.com/p/gdata-java-client/source/browse/trunk/java/sample/oauth/TwoLeggedOAuthExample.java
From
To retrieve all groups in a particular domain, use the following GET request:
GET [https://apps-apis.google.com/a/feeds/group/2.0/domain[?[start|https://apps-apis.google.com/a/feeds/group/2.0/domain[?[start]=]]
To retrieve all the groups for a particular member [in a domain], use the following GET request:
GET [https://apps-apis.google.com/a/feeds/group/2.0/domain/?member=memberId[&directOnly=true|false|https://apps-apis.google.com/a/feeds/group/2.0/domain/?member=memberId[&directOnly=true]]
directOnly - If true, the request identifies only members with direct association with the group.
Tried the first query, i.e.
GET https://apps-apis.google.com/a/feeds/group/2.0/sagebionetworks.com
with these request properties:
conn.setRequestProperty("Authorization: OAuth",
"oauth_consumer_key=sandbox-sagebionetworks.appspot.com, "
"oauth_nonce="+nonce()", "
+"oauth_signature_method=HMAC-SHA1, "
+"oauth_signature=OrB7tgg7BJtAkn9uvbo14uC9"
+"oauth_timestamp="+System.currentTimeMillis()
);
Result: Got 401 OK
:^(
Added xoauth_requestor_id param in the URL
Result: Got 401 OK
:^(
Will try Google example from here:
http://code.google.com/p/gdata-java-client/source/browse/trunk/java/sample/oauth/TwoLeggedOAuthExample.java
Went here to get the Java API
http://code.google.com/p/google-api-java-client/
downloaded the file
google-api-client-1.2.1-alpha-java.zip
Next steps:
1. Complete the installation
2. Get the google example running
Need google data java client:
http://code.google.com/apis/gdata/articles/java_client_lib.html
Chose this
http://code.google.com/p/gdata-java-client/downloads/detail?name=gdata-src.java-1.42.0.zip&can=2&q=
extracted the file to c:/users/hoff/eclipse
there are a lot of files, including source
ran the project but got 'classnotfound' exceptions
had to copy files to war/web-inf/lib in addition to adding them to the project as external jars
got
java.lang.NoClassDefFoundError: com/google/common/collect/Maps
and put directly into the war/web-inf/lib (not build path)
app now runs.
Fixed a couple nullptr exceptions and it ran w/o error.
Restored the security-constraint to the web.xml file (accidentally erased).
Deployed to GAE.
It works! I had to iterate on the code example to get the group information out,
but eventually saw the two groups 'demo' and 'test' in the sagebionetworks.com domain.
Next step: retrieve group membership for a user.
Done: The authorization demo now runs -- comparing user's groups to those authorized for a service.
Authenticate and Authorize a web application directly against Crowd
The approach taken above is 'tightly integrated' with Google Apps since any user accessing any resource (even a non-Google App resource) needs to have a Google Apps account. It would be helpful if users could authenticate against Crowd directly, even without having a Google Apps account on our domain. The next phase pursues this approach:
First I tried implementing the Crowd Java Client, in a GAE application. This turned out to be a dead end, but I include my notes anyway:
Implementing the Crowd Java Client
notes from implementing Crowd demo:
1. Moved the unzipped 'atlassian-crowd-2.1.0' under my 'eclipse projects' folder
2. Created a new java project from the folder atlassian-crowd-2.1.0/demo-src
3. Add external jar files from atlassian-crowd-2.1.0/client/
crowd-integration-client.2.1.0.jar
lib/*.jar
Result: There are build errors:
Logoff.java won't compile, with messages about HttpServlet... missing
The project is missing the J2EE stuff. I think the GAE projects enjoy
having servlet libraries wrapped up with other google libraries.
This seems to be the authentication API:
Looks like a way to integrate as a "Filter"
http://docs.atlassian.com/crowd/2.1.0-beta2/index.html?com/atlassian/crowd/integration/http/CrowdHttpAuthenticator.html
Here's a demo for getting the 'principals' for a Group:
It would be 'slicker' to find the groups for a principal rather than the principals for a group.
--------------------------------------
From:
http://confluence.atlassian.com/display/CROWDDEV/Java+Integration+Libraries
VerifyTokenFilter -- looks like what I want.
To try out Crowd authentication:
1. Split the GAE demo into a 'GoogleApps' section and a 'Crowd' section
2. Added the VerifyTokenFilter to web.xml (just for the 'Crowd' secton of the app)
3. Copied eclipse projects/atlassian-crowd-2.1.0/crowd-integration-client-2.1.0.jar to war/WEB-INF/lib
at this point I tried running the app but got
java.lang.NoClassDefFoundError: org/apache/log4j/Logger
4. Took the hint and added all the file in
atlassian-crowd-2.1.0/client/lib
to war/WEB-INF/lib
Tried running again, but got
java.lang.NoClassDefFoundError: java.rmi.server.UID is a restricted class.
Please see the Google App Engine developer's guide for more details.
after VerifyTokenFilter indirectly called it.
According to this
http://jira.atlassian.com/browse/CWD-1748
there is an additional issue, which is that the Crowd client tries to spawn threads,
not allowed by GAE.
So I may have to reimplement the authentication using the ('restful') web api.
Implementing the Crowd RESTful API
The documentation starts here:
http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+APIs
I should try this example:
http://YOUR-CROWD-SERVER:8095/rest/usermanagement/1/user?username=USERNAME
http://deflaux:8095/rest/usermanagement/1/user?username=mike.kellen
returns 404 error, probably because I have not included the application credentials in the request.
From
http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+Resources
This looks like the way to find if a user is a member of a group:
/user/group/nested?username=USERNAME&groupname=GROUPNAME
near the bottom:
Crowd SSO Token Resource
seems to give the methods for authenticating the user.
I believe I need credentials for making the request.
I logged in to the crowd admin console.
I clicked 'add application' and entered the following parameters:
Application Type: Generic Application
name: gae-client
description: gateway filter for authentication against crowd
password: gae-client-pw
<next page>
url: http://sandbox-sagebionetworks.appspot.com/
<Clicked 'Resolve IP Address', which filled in the next field:>
Remote IP Address: 127.0.0.1
(I don't get this!)
<next page>
Directories:
Evaluation
Authorisation:
Clicked "Let all users in this directory authenticate against the gae-client application."
*DONE*
Tried writing a class 'CrowdAuthUtil' to isse the RESTful request to get a given user
It gives me the 404 error. I have to incude the credentials.
Based on my reading, I should be getting a '401' response, to challenge me for the credentials.
I may have to run it on GAE to get this far. I changed 'deflaux' to '140.107.149.214'
so GAE can find Crowd.
Got
Timeout while fetching: http://140.107.149.214:8095/rest/usermanagement/1/user?username=mike.kellen
maybe I need to host the Crowd server elsewhere!
From
http://confluence.atlassian.com/display/CROWDDEV/Using+the+Crowd+REST+APIs
Access to all resources (using any method) requires the client to be
authenticated via basic authentication. See RFC 2617.
http://frontier.userland.com/stories/storyReader$2159
Suggests I should be getting an authentication challenge when I issue an HTTP Request.
Found a Base64 encoder here:
http://sourceforge.net/projects/iharder/files/base64/2.3/
and added the basic authorization property to the http request.
This guy has the same problem as me:
http://forums.atlassian.com/thread.jspa?messageID=257358182
Need to debug this... downloaded cURL from here:
http://curl.haxx.se/latest.cgi?curl=win64-ssl-sspi
The manual is here:
http://curl.haxx.se/docs/manual.html
curl -u gae-client:gae-client-pw http://140.107.149.214:8095/rest/usermanagement/1/user?username=mike.kellen
returns the same 404 error that I saw in my web browser
curl -u gae-client:gae-client-pw -H "Accept: application/xml" http://deflaux:8095/rest/usermanagement/1/user?username=mike.kellen
same result!
I sent this on 12/14
Dear Sirs,
I have a demo version of Crowd up and running and have two questions:
1) When I try to exercise the Crowd rest api I get a "404" error from Tomcat. Specifically in cURL I send
curl -u a-client:a-client-pw -H "Accept: application/xml" [http://<crowd (http://%3ccrowd/)] server addr>:8095/rest/usermanagement/1/user?username=<a user>
where <crowd server addr> is the location of my server and <a user> is a user I set up in the evaluation directory.
"a-client" and "a-client-pw" are the client name and pw I set up for a generic application in the Crowd admin' utility.
When I execute this http request I get back
<html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><!--H1 {fon t-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size: 22px;}H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525 D76;font-size:16px;}H3 {font-family:Tahoma,Arial,sans-serif;color:white;backgro und-color:#525D76;font-size:14px;}BODY {font-family:Tahoma,Arial,sans-serif;col or:black;background-color:white;}B {font-family:Tahoma,Arial,sans-serif;color:w hite;background-color:#525D76;}P {font-family:Tahoma,Arial,sans-serif;backgroun d:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR
{color : #525D76;}--></style> </head><body><h1>HTTP Status 404 - /rest/usermanag
ement/1/user</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p>
<p><b>message</b> <u>/rest/usermanagement/1/user</u></p><p><b>description</b> <u
>The requested resource (/rest/usermanagement/1/user) is not available.</u></p><
HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.26</h3></body></html>
Can you tell me where I went wrong? Is there something specific I need to do to enable the rest API beyond starting up Crowd?
2) If I use the rest api along with the SAML-based Google App's connector, will I get single sign-on across
all applications? That is, if I use the rest api's SSO resource to sign-on, how does the Google App's connector
"know" that I'm signed on and avoid re-authenticating? Is there something special I need to do with the
token returned by the SSO resource so that the Google App's connector "finds" it?
Thanks.
Response:
Q: Can you tell me where I went wrong? Is there something specific I need to do to enable the rest API beyond starting up Crowd?
A: You don't need to do anything special to start up the REST API. I would recommend using the syntax containing the context
(e.g. [http://<CROWD_SERVER>:8095/crowd/rest/usermanagement/latest/user?username=USERNAME (http://%3ccrowd_server%3e:8095/crowd/rest/usermanagement/latest/user?username=USERNAME)]).
Also, remember that the REST API if for applications to access Crowd. You should use the application name and password created when
you set up a custom application as your credentials for the REST service ("a-client" and "a-client-pw") in your example.
Q: If I use the rest api along with the SAML-based Google App's connector, will I get single sign-on across all applications?
A: SSO with Google plus one or more Crowd-connected applications works. Google uses it's own mechanism, however and does not use
Crowd's SSO cookie/token. Details on how SSO works in Crowd can be found here.
http://confluence.atlassian.com/display/CROWD/Overview+of+SSO
---------------------------------
I'm retrying the query using context='crowd':
curl -u gae-client:gae-client-pw -H "Accept: application/xml" http://deflaux:8095/crowd/rest/usermanagement/latest/user?username=mike.kellen
This 'works', i.e. I now reach the service. But it says I'm not authorized to access it due to my IP address.
This is expected. I added 140.107.176.92 as an allowed 'remote address' in the Crowd Applications tab > Remote Addresses
It works! I get back:
<user name="mike.kellen" expand="attributes">
<link rel="self" href="http://deflaux:8095/crowd/rest/usermanagement/latest/user?username=mike.kellen"/>
<first-name>Mike</first-name>
<last-name>Kellen</last-name>
<display-name>Mike Kellen</display-name>
<email>bruce.hoff@sagebase.org</email>
<password>
<link rel="edit" href="http://deflaux:8095/crowd/rest/usermanagement/latest/user/password?username=mike.kellen"/>
</password>
<active>true</active>
<attributes>
<link rel="self" href="http://deflaux:8095/crowd/rest/usermanagement/latest/user/attribute?username=mike.kellen"/>
</attributes>
</user>
<error>
<reason>MEMBERSHIP_NOT_FOUND</reason>
-
<message>
The child entity <mike.kellen> is not a member of the parent <demo>
</message>
</error>
created a group in Crowd called 'demo-group' and added Mike.
http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=mike.kellen&groupname=demo-group
returns
<group name="demo-group">
<link rel="self" href="http://deflaux:8095/crowd/rest/usermanagement/latest/group?groupname=demo-group"/>
</group>
but
http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=bruce.hoff&groupname=demo-group
returns
<error>
<reason>MEMBERSHIP_NOT_FOUND</reason>
-
<message>
The child entity <bruce.hoff> is not a member of the parent <demo-group>
</message>
</error>
Looks great. Now we need to exercise SSO. Per
http://confluence.atlassian.com/display/CROWDDEV/Crowd+REST+Resources
we POST
http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false
with the content
<?xml version="1.0" encoding="UTF-8 "?>
<authentication-context>
<username>mike.kellen</username>
<password></password>
<validation-factors>
<validation-factor>
<name>remote_address</name>
<value>127.0.0.1</value>
</validation-factor>
</validation-factors>
</authentication-context>
In cURL:
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -d "%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%20%20%0D%0A%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20" http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false
returned a 500 error
curl -u gae-client:gae-client-pw -H "Accept: application/xml" http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><error><reason>INVALID_SSO_TOKEN</reason>
<message>Failed to find entity of type [com.atlassian.crowd.model.token.Token]
with identifier [validate-password=false]</message></error>
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -d "%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20%20" http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false
again returns a 500
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -d "<?xml version="1.0" encoding="UTF-8 "?> <authentication-context> <username>mike.kellen</username> <password></password> <validation-factors> <validation-factor> <name>remote_address</name> <value>127.0.0.1</value> </validation-factor> </validation-factors> </authentication-context> " http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false
again returns a 500
curl -u gae-client:gae-client-pw -H "Accept: application/xml" --data "%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%20%20%0D%0A%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20" http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false
again returns a 500
I don't see where you specify the HTTP method, but this example seems to say it's with the -X option
http://groups.google.com/group/brightkite-api/browse_thread/thread/b5d8402ed6f3ca69
curl -X POST -u gae-client:gae-client-pw -H "Accept: application/xml" -d "%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%20%20%0D%0A%3Cauthentication-context%3E%20%20%0D%0A%20%20%3Cusername%3Emike.kellen%3C%2Fusername%3E%20%20%0D%0A%20%20%3Cpassword%3E%3C%2Fpassword%3E%20%20%0D%0A%20%20%3Cvalidation-factors%3E%20%20%0D%0A%20%20%20%20%3Cvalidation-factor%3E%20%20%0D%0A%20%20%20%20%20%20%3Cname%3Eremote_address%3C%2Fname%3E%20%20%0D%0A%20%20%20%20%20%20%3Cvalue%3E127.0.0.1%3C%2Fvalue%3E%20%20%0D%0A%20%20%20%20%3C%2Fvalidation-factor%3E%20%20%0D%0A%20%20%3C%2Fvalidation-factors%3E%20%20%0D%0A%3C%2Fauthentication-context%3E%20" http://deflaux:8095/crowd/rest/usermanagement/latest/session/validate-password=false
Once I figure this out, I have to handle the cookie, perhaps as explained here:
http://www.hccp.org/java-net-cookie-how-to.html
... then I have to get it running on GAE, which may mean rehosting the Crowd server.
One note: It looks like Crowd groups don't span multiple directories. Need to check this.
curl -u gae-client:gae-client-pw -H "Accept: application/xml" http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/nested?username=bruce.hoff
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?>
<groups expand="group">
<group name="crowd-administrators">
<link rel="self" href="http://deflaux:8095/crowd/rest/usermanagement/latest/group?groupname=crowd-administrators"/>
</group>
</groups>
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group
Returns
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><status><status-code>500</status-code></status>* Closing connection #0
'groupname' is not recognized as an internal or external command, operable program or batch file.
... so it seems to be complaining about the 'groupname' param tacked on the end of the URL
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff
returns
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?>
<status><status-code>500</status-code></status>
I keep getting the '500' error while trying to use the Crowd REST API. I finally sent another product enquiry to Atlassian with an example:
I get errors using some of the Crowd REST API commands. An example is adding a user to a group. Below I give the cURL command, the result, and the stack trace in the Tomcat log file. Can anyone say whether this is a Crowd bug or what I've done wrong and, if I've done something wrong, what the correction is? Thanks!
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group
* About to connect() to deflaux port 8095 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8095 (#0)
* Server auth using Basic with user 'gae-client'
> POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8095
> Accept: application/xml
> Content-Length: 66
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 500 Internal Server Error
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Set-Cookie: JSESSIONID=6A8AF73DC2298816307074EC949B18FE; Path=/crowd
< Cache-Control: no-cache, no-store, no-transform
< Content-Type: application/xml
< Content-Length: 102
< Date: Fri, 17 Dec 2010 23:42:41 GMT
< Connection: close
<
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><status><status-code>500</status-code></status>* Closing connection #0
'groupname' is not recognized as an internal or external command,
operable program or batch file.
from tomcat/apache-tomcat/logs/cataline.2010-12-17.log
Dec 17, 2010 3:42:41 PM com.sun.jersey.server.impl.application.WebApplicationImpl onException
SEVERE: Internal server error
javax.ws.rs.WebApplicationException
at com.sun.jersey.server.impl.uri.rules.TerminatingRule.accept(TerminatingRule.java:55)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:654)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:612)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:603)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:309)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:425)
at com.atlassian.plugins.rest.module.RestDelegatingServletFilter$JerseyOsgiServletContainer.doFilter(RestDelegatingServletFilter.java:203)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:689)
at com.atlassian.plugins.rest.module.RestDelegatingServletFilter.doFilter(RestDelegatingServletFilter.java:112)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
at com.atlassian.plugins.rest.module.servlet.RestServletUtilsUpdaterFilter.doFilterInternal(RestServletUtilsUpdaterFilter.java:26)
at com.atlassian.plugins.rest.module.servlet.RestServletUtilsUpdaterFilter.doFilter(RestServletUtilsUpdaterFilter.java:40)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:169)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at com.atlassian.crowd.console.filter.CrowdDelegatingFilterProxy.doFilter(CrowdDelegatingFilterProxy.java:38)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
at com.atlassian.crowd.plugin.rest.auth.BasicApplicationAuthenticationFilter.doFilter(BasicApplicationAuthenticationFilter.java:80)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)
at com.atlassian.crowd.console.filter.CrowdOpenSessionInViewFilter.doFilterInternal(CrowdOpenSessionInViewFilter.java:26)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
at com.atlassian.crowd.plugin.rest.filter.CrowdVersionFilter.doFilter(CrowdVersionFilter.java:39)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:55)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:41)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:636)
TIME PASSES....
After some exchanges with Atlassian, it seems the main problem is that an additional header is needed in the HTTP Request, "Content-Type: application/xml". Here're the latest results:
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>mike.kellen</username> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.136</value> </validation-factor> </validation-factors> </authentication-context>" -X POST http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false
* About to connect() to deflaux port 8095 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8095 (#0)
* Server auth using Basic with user 'gae-client'
> POST /crowd/rest/usermanagement/1/session?validate-password=false HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8095
> Accept: application/xml
> Content-Type: application/xml
> Content-Length: 265
>
< HTTP/1.1 400 Bad Request
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Set-Cookie: JSESSIONID=18A78725E7376FF298A2FAB502A99DD3; Path=/crowd
< Content-Type: application/xml
< Content-Length: 150
< Date: Wed, 22 Dec 2010 17:19:00 GMT
< Connection: close
<
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><error><reason>ILLEGAL_ARGUMENT</reason><message>The validated object is null</message></error>* Closing connection #0
Updated the server IP address...
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>mike.kellen</username> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.214</value> </validation-factor> </validation-factors> </authentication-context>" -X POST http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false
... but got the same result. Try empty password field...
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>mike.kellen</username> <password></password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.214</value> </validation-factor> </validation-factors> </authentication-context>" -X POST http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false
... but got same result. Try filled in p/w field...
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>mike.kellen</username> <password>ssotest</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.149.214</value> </validation-factor> </validation-factors> </authentication-context>" -X POST http://deflaux:8095/crowd/rest/usermanagement/1/session?validate-password=false
... same result. I sent another request to Atlassian to ask about the ILLEGAL_ARGUMENT error.
Back to the add-user-to-group command. The original command was:
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group
which generated 500 error. Adding in 'content-type' header:
curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group
results in 201 'created' message, but there's still the error "'groupname' is not recognized...".
* About to connect() to deflaux port 8095 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8095 (#0)
* Server auth using Basic with user 'gae-client'
> POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8095
> Content-Type: application/xml
> Accept: application/xml
> Content-Length: 65
>
< HTTP/1.1 201 Created
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Set-Cookie: JSESSIONID=27110347D1CD6EBAEBCD42CA6673DD57; Path=/crowd
< Location: http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.ho
ff&groupname=demo-group
< Content-Type: application/xml
< Content-Length: 0
< Date: Wed, 22 Dec 2010 17:43:27 GMT
<
* Connection #0 to host deflaux left intact
* Closing connection #0
'groupname' is not recognized as an internal or external command,
operable program or batch file.
... putting quotes around URL...
curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
... works perfectly:
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: appli
cation/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "http:
//deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo
-group"
* About to connect() to deflaux port 8095 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8095 (#0)
* Server auth using Basic with user 'gae-client'
> POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group
HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8095
> Content-Type: application/xml
> Accept: application/xml
> Content-Length: 65
>
< HTTP/1.1 201 Created
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Set-Cookie: JSESSIONID=CB8B969B95FFA3F6638CC7279DC7FD45; Path=/crowd
< Location: http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.ho
ff&groupname=demo-group
< Content-Type: application/xml
< Content-Length: 0
< Date: Wed, 22 Dec 2010 17:51:40 GMT
<
* Connection #0 to host deflaux left intact
* Closing connection #0
Moreover I can log in to the Crowd admin console and see that the user is added to the group.
Crowd + HTTPS
The sticking point above is that the SSO method for gettng a session token requires the user name and PASSWORD to be sent, which is not secure. To make the use of the Crowd REST API secure we want to send the request via HTTPS rather than HTTP:
We enabled https on the demo crowd server by following the instructions here:
http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL
Now we exercise the API:
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>mike.kellen</username> <password>ssotest</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentication-context>" -X POST https://deflaux:8443/crowd/rest/usermanagement/1/session?validate-password=true
generates 403 message
The add-user-to-group command no longer works:
curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
* About to connect() to deflaux port 8443 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS alert, Server hello (2):
* SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
* Closing connection #0
curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "ht
tps://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
* About to connect() to deflaux port 8443 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* start date: 2011-02-03 00:06:18 GMT
* expire date: 2011-05-04 00:06:18 GMT
* common name: Bruce Hoff (does not match 'deflaux')
* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* Server auth using Basic with user 'gae-client'
> POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8443
> Content-Type: application/xml
> Accept: application/xml
> Content-Length: 65
>
< HTTP/1.1 403 Forbidden
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Content-Type: text/html;charset=utf-8
< Content-Length: 1429
< Date: Thu, 03 Feb 2011 22:18:10 GMT
<
<html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><!--H1
H2
H3
BODY
B
P
A
A.name
HR
--></style> </head><body><h1>HTTP Status 403 - Client with address "140.107.149.249", and hostname "140.107.149.249" is fo
rbidden from making requests to the application, gae-client.</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Client with address "140.107.149.249", an
d hostname "140.107.149.249" is forbidden from making requests to the application, gae-client.</u></p><p><b>description</b> <u>Access to the specified resource (Client with address "140
.107.149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, gae-client.) has been forbidden.</u></p><HR size="1" noshade="noshade"><h3>Apache Tom
cat/6.0.26</h3></body></html>* Connection #0 to host deflaux left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
Tried adding 140.107/16 to list of allowed addresses, but that didn't help.
Tried adding 140.107.149.249, which WORKS!!!
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "ht
tps://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
* About to connect() to deflaux port 8443 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* start date: 2011-02-03 00:06:18 GMT
* expire date: 2011-05-04 00:06:18 GMT
* common name: Bruce Hoff (does not match 'deflaux')
* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* Server auth using Basic with user 'gae-client'
> POST /crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8443
> Content-Type: application/xml
> Accept: application/xml
> Content-Length: 65
>
< HTTP/1.1 201 Created
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Set-Cookie: JSESSIONID=B875CF7183833A03C4E8CF6DFC0243B4; Path=/crowd; Secure
< Location: https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group
< Content-Type: application/xml
< Content-Length: 0
< Date: Thu, 03 Feb 2011 22:21:41 GMT
<
* Connection #0 to host deflaux left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "https://deflaux:8443/crowd/rest/usermanagement/latest/user/group/direct?username=john.hill&groupname=demo-group"
Now that I added my current IP addr into the trusted list, the 'session' command works:
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>mike
.kellen</username> <password>ssotest</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authent
ication-context>" -X POST https://deflaux:8443/crowd/rest/usermanagement/1/session?validate-password=true
* About to connect() to deflaux port 8443 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* start date: 2011-02-03 00:06:18 GMT
* expire date: 2011-05-04 00:06:18 GMT
* common name: Bruce Hoff (does not match 'deflaux')
* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* Server auth using Basic with user 'gae-client'
> POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8443
> Accept: application/xml
> Content-Type: application/xml
> Content-Length: 294
>
< HTTP/1.1 201 Created
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Set-Cookie: JSESSIONID=F8B3FE2A89EBD0A545D16C67EA658FC5; Path=/crowd; Secure
< Cache-Control: no-cache, no-store, no-transform
< Location: https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00
< Content-Type: application/xml
< Content-Length: 366
< Date: Thu, 03 Feb 2011 22:33:43 GMT
<
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><session expand="user"><token>JBjiPXiXVnqgEg8cpn2kWg00</token><user name="mike.kellen"><link rel="self" href="https://deflaux:8443/crowd/rest/use
rmanagement/1/user?username=mike.kellen"/></user><link rel="self" href="https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00"/></session>* Connection #0 to host deflaux le
ft intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
Try validating an existing token:
curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -X GET https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -X GET https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn
2kWg00
* About to connect() to deflaux port 8443 (#0)
* Trying 140.107.149.214... connected
* Connected to deflaux (140.107.149.214) port 8443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* start date: 2011-02-03 00:06:18 GMT
* expire date: 2011-05-04 00:06:18 GMT
* common name: Bruce Hoff (does not match 'deflaux')
* issuer: C=US; ST=WA; L=Seattle; O=Sage; OU=Platform; CN=Bruce Hoff
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* Server auth using Basic with user 'gae-client'
> GET /crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00 HTTP/1.1
> Authorization: Basic Z2FlLWNsaWVudDpnYWUtY2xpZW50LXB3
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: deflaux:8443
> Accept: application/xml
> Content-Type: application/xml
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.0
< Set-Cookie: JSESSIONID=8E644012E156354E691D22B7C616D2CB; Path=/crowd; Secure
< Content-Type: application/xml
< Content-Length: 788
< Date: Thu, 03 Feb 2011 22:39:23 GMT
<
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><session expand="user"><token>JBjiPXiXVnqgEg8cpn2kWg00</token><user name="mike.kellen"><link rel="self" href="https://deflaux:8443/crowd/rest/use
rmanagement/1/user?username=mike.kellen"/><first-name>Mike</first-name><last-name>Kellen</last-name><display-name>Mike Kellen</display-name><email>bruce.hoff@sagebase.org</email><password><link rel="e
dit" href="[https://deflaux:8443/crowd/rest/usermanagement/1/user/password?username=mike.kellen"/></password><active>true</active><attributes><link (https://deflaux:8443/crowd/rest/usermanagement/1/user/password?username=mike.kellen"></a></password><active>true</active><attributes><link)] rel="self" href="https://deflaux:8443/crowd/rest/user
management/1/user/attribute?username=mike.kellen"/></attributes></user><link rel="self" href="https://deflaux:8443/crowd/rest/usermanagement/1/session/JBjiPXiXVnqgEg8cpn2kWg00"/></session>* Connection
#0 to host deflaux left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
We also verified that we can force a token to expire.
___________________________________________________________________________________
Tried the original command (worked before SSO enable):
curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "http://deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
C:\Users\bhoff>curl -u gae-client:gae-client-pw -H "Content-Type: application/xml" -H "Accept: application/xml" -v -d "<?xml version='1.0' encoding='UTF-8 '?> <group name='demo-group'/>" -X POST "http:
//deflaux:8095/crowd/rest/usermanagement/latest/user/group/direct?username=bruce.hoff&groupname=demo-group"
* About to connect() to deflaux port 8095 (#0)
* Trying 140.107.149.214... Connection refused
* couldn't connect to host
* Closing connection #0
curl: (7) couldn't connect to host
So while enabling HTTPS we have also disabled HTTP.
__________________________________________________________________
Notes
Q: What's the cumulative file size on the Sage SSH server?
A: About 2GB, considering the files in the directory /data/incoming on sage.fhcrc.org
Google Apps provides two APIs to help with authentication:
SAML Single Sign-On (SSO) Service: would allow *us* to create and maintain users and groups outside of Google.
http://code.google.com/googleapps/domain/sso/saml_reference_implementation.html
Google Apps Provisioning API: would allow us to programmatically create Google users and groups in our private domain. This would streamline adding users to Google Apps. If we used it as a total solution, then the non-google app's (e.g. Addama) would have to go to google for authentication, which violates the 'arms length' integration requirement.
OpenID sounds like an alternative to SAML:
http://www.google.com/support/forum/p/apps-apis/thread?tid=33a3707bd2ea7904&hl=en
In the case of OpenID, the user may have a Google Account, a Google Apps Account, or an account from any other domain that provides OpenID federated login.
Integration of GAE with OpenID:
http://code.google.com/appengine/docs/java/users/overview.html
"What would Atlassian Do" (WWAD)?
4.1 Seraph is a very simple, pluggable J2EE web application security framework developed by Atlassian and used in our products.
http://confluence.atlassian.com/display/DEV/Single+Sign-on+Integration+with+JIRA+and+Confluence
4.2 Crowd is a single sign-on (SSO) application for as many users, web applications
and directory servers you need — all through a single web interface.
http://www.atlassian.com/software/crowd/
Crowd centralises identity management, allowing you to take users from different directories
and manage them in one place. Multiple user directories can be centrally managed via Crowd's
administration console.
Crowd's OpenID authentication server, CrowdID, talks with websites and applications using
OpenID. It expands Crowd's SSO capabilities to applications outside your organisation's firewall.
http://confluence.atlassian.com/display/CROWD/Configuring+the+Google+Apps+Connector
To enable single sign-on in Google Apps, you will need the Premier, Education, or Partners edition of Google Apps.
The Crowd Google Apps connector does not support the automatic adding of users. If a user exists
in Crowd but not in Google Apps, then the user will not be able to log in to Google Apps.
To add an application (e.g. a GAE app like Addama registry):
http://confluence.atlassian.com/display/CROWDDEV/Application+Integration+Overview
Licensing and hosting Crowd:
- Crowd is not hosted by Atlassian. We have to run it ourselves. It runs on Windows, Linux or Mac and uses an apache tomcat app server:
http://confluence.atlassian.com/display/CROWD/Installing+Crowd+and+CrowdID
- Pricing: $10 for up to 10 users then $600/$1200 for up to 100 users (academic/commercial) and $4000/$8000 for unlimited users.
http://www.atlassian.com/software/crowd/pricing.jsp
Also the user count is the 'total number of users who can authenticate against the Crowd instance', not just the number of simultaneously active users. From
http://www.atlassian.com/software/crowd/licensing.jsp#terms
"Licensing fees are quoted per total number of 'Crowd users'. A Crowd user is defined as any user account that can authenticate against one or more applications."
To integrate other applications with Crowd:
Writing a Crowd custom application connector:
http://confluence.atlassian.com/display/CROWDDEV/Creating+a+Crowd+Client+for+your+Custom+Application
To integrate with Apache's authentication:
http://confluence.atlassian.com/display/CROWD/Integrating+Crowd+with+Apache
Other Single Sign-On software:
Another alternative is "SSO Easy".
Open source alternative:
http://code.google.com/googleapps/domain/open_source_projects.html#sso
Enterprise Sign-on Engine:
http://esoeproject.org/
Other notes
- Sage SSH/SCP server authenticates using standard unix log-in.
Addama Authentication/Authorization
Some info here
http://code.google.com/p/addama/w/list
- Addama authentication is via Servlet filters using GAE User Service OR a Google API-key.
- Addama handles authentication via Servlet Filters; the servlet config xml file shows what's in place.
- Addama white list: "user x can get these services, or anything under the branch."
Notes on Addama Registry Filters:
org.systemsbiology.addama.coresvcs.gae.filters.StaticContentFilter
I don't think this has anything to do with authentication, rather it's a cache for static content.
Note: You can't even get this far without being authenticated.
Note: The white list (below) *authorizes*, and doesn't apply to static content.
org.systemsbiology.addama.coresvcs.gae.filters.UserServiceFilter
If logged-in Google Acct OR valid API Key, then allow, else deny.
org.systemsbiology.addama.coresvcs.gae.filters.WhiteListFilter
If the user is an Admin or is in a 'white list' for the requested resource, then allow, else deny.
org.systemsbiology.addama.coresvcs.gae.filters.DirectLinkFilter
Seems to handle a specific kind of request called a 'direct link' request.
(This MIGHT be a method for retrieving large files.)
org.systemsbiology.addama.coresvcs.gae.filters.AdminOnlyFilter
Filter out any requests NOT from an admin.
Applied only for addama/memcache/*
org.systemsbiology.addama.coresvcs.gae.filters.ProxiesFilter
Seems to forward certain requests (in particular, non-registry requests) to GAE's "URLFetchService".
- what does <security-constraint> in the GAE web.xml file mean?
A: from http://code.google.com/appengine/docs/java/users/overview.html
If you have pages that the user should not be able to access unless signed in, you can establish a security constraint for those pages in the deployment descriptor (the web.xml or app.yaml file). If a user accesses a URL with a security constraint and the user is not signed in, App Engine redirects the user to the sign-in page automatically (for Google Accounts or Google Apps authentication) or to the page at /_ah/login_required (for OpenID authentication), then directs the user back to the URL after signing in or registering successfully.
A security constraint can also require that the user be a registered administrator for the application. This makes it easy to build administrator-only sections of the site, without having to implement a separate authorization mechanism.
Nicole's "an area for testing" is a "google apps for your domain" domain
http://www.google.com/a/sagebionetworks.com is a "test domain for Google Apps"
What's the difference between a "google account" and a "google apps account"?
A: the latter is newer and ultimately should subsume the former.
Does Google Apps support OpenID?
A: Only as an "Identity provider" (of the Google Apps ID) not as a service provider seeking authentication.
http://code.google.com/googleapps/domain/sso/openid_reference_implementation.html
3 ways to authenticate GAE
- google accounts
- google-apps account (on proprietary domain associated with Google)
- OpenID
ours is a google apps premier (="business"?) account
Notes on Authorization
From:
http://www.google.com/support/a/bin/answer.py?answer=61017&hl=en
With Google Apps for Business and Education, administrators can use two-legged OAuth for
domain-wide delegation of authority. An application that has the OAuth consumer key and
secret (roughly equivalent to a role account username and password) is allowed to act as
any user in the domain when accessing Google Data APIs. Unlike three-legged OAuth, users
do not need to give consent on an individual basis, as this decision is made on their behalf
by the administrator.
This is one of the admin screens in sagebionetworks.com for authorization:
https://www.google.com/a/cpanel/sagebionetworks.com/ManageOauthClients
From:
http://www.google.com/support/forum/p/apps-apis/thread?tid=21d4f55e584e3970&hl=en
Also, for access control, your SSO solution can create a cookie for the user storing
his username. When a specific website page is requested, you can again use Provisioning
API to determine if the username stored in cookie is part of the required Google group
and authorize him.
A Dead End
I starting implementing OAuth, but mistakenly started working on '3 legged OAuth' when in fact I wanted to implenent the much simpler '2 legged' variety. I include my notes in case we have to go back to '3 legged authorization later.'
Went to
http://code.google.com/apis/accounts/docs/OAuth.html#GoogleAppsOAuth
which says that I have to do the following sequence:
1. Get an unauthorized request token (OAuthGetRequestToken)
2. Authorize the request token (OAuthAuthorizeToken)
3. Exchange the authorized request token for an access token (OAuthGetAccessToken)
1. issue this request:
https://www.google.com/accounts/OAuthGetRequestToken
with query param's (also can go elsewhere, values are my best guesses right now):
oauth_consumer_key=sandbox-sagebionetworks.appspot.com
oauth_nonce= ""+Math.abs(Random.nextLong()) // (must be unsigned)
oauth_signature_method=HMAC-SHA1
oauth_signature=OrB7tgg7BJtAkn9uvbo14uC9
oauth_timestamp=System.currentTimeMillis()
scope=https://apps-apis.google.com/a/feeds/group/#readonly https://apps-apis.google.com/a/feeds/user/#readonly
oauth_callback= a URL our application supplies to receive a <request token, verifier> pair
The response header includes (for example)
oauth_token=ab3cd9j4ks73hf7g
&oauth_token_secret=ZXhhbXBsZS5jb20
&oauth_callback_confirmed=true
2. Issue this request:
https://www.google.com/accounts/OAuthAuthorizeToken
oauth_token=<the value returned above>
hd=sagebionetworks.com
Now our callback URL is called to receive the <reques token, verifier> pair
3. Issue this request:
https://www.google.com/accounts/OAuthGetAccessToken
oauth_consumer_key=sandbox-sagebionetworks.appspot.com
oauth_token=<the request token returned from request #1>
oauth_verifier=<verification code supplied in the callback>
oauth_signature_method=HMAC-SHA1
oauth_signature=OrB7tgg7BJtAkn9uvbo14uC9
oauth_timestamp=System.currentTimeMillis()
oauth_nonce= ""+Math.abs(Random.nextLong()) (must be unsigned)
Response header contains the access code
oauth_token=ab3cd9j4ks73hf7g&oauth_token_secret=ZXhhbXBsZS5jb20
I wrote a routine to do step 1, with all param's except 'scope' and 'auth_callback'.
When I ran it, it returned: "400 OK"
Changed the app to have valid values for the scop and callback. Tried again. Result:
Result:
java.io.IOException: Could not fetch URL: https://www.google.com/accounts/OAuthGetRequestToken....
I think it's a matter of formatting the param's.
I moved the params into the request header. Now the IOException goes away and I'm back to 400/OK.
restarted server
initially there is no public DNS
after a few minutes it's running, with the public DNS name:
ec2-50-16-44-152.compute-1.amazonaws.com
totally different from before!
Tried going tohttp://ec2-50-16-44-152.compute-1.amazonaws.com:8095/crowd/console
but it failed
PuTTY'd into the server:
Ran 'ps', but it showed no Java process running, so I think the server is down.
cd /mnt/ebs successfully, so the ebs seems to still be running.
However, there are no files there and 'df' doesn't show the storage.
So I think the mount was lost.
The AWS console shows that the EBS volume is 'in use' and attached to the instance (i-c31579af:/dev/sdh)
Back in PuTTY:
sudo mount /dev/sdh /mnt/ebs
Now I see the 10GB of space, but the content is gone(!)
Tried detaching and reattaching in the AWS console.
Didn't work. I give up!
___________________
Started over, using my sage credentials
- created "Security Group" in the AWS console
with HTTP port 8095 and HTTPS port 8443.
- created a 'micro' instance
"Public DNS" is:
ec2-50-17-17-19.compute-1.amazonaws.com
copied atlassian-crowd-2.1.1.tar.gz to /home/ec2-user
gunzip atlassian-crowd-2.1.1.tar.gz
sudo mv atlassian-crowd-2.1.1.tar /usr/local
sudo tar -xvf atlassian-crowd-2.1.1.tar
sudo chmod -R 777 atlassian-crowd-2.1.1
edited crowd-webapp/WEB-INF/classes/crowd-init.properties
just uncommented the default so
crowd.home=/var/crowd-home
./start_crowd.sh
http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd
resulted in 404 error. So tomcat is running (and listening on port 8095), but Crowd isn't
My guess is that it's a permissions problem.
sudo chmod -R 777 var
stopped/restarted Crowd
Now it works!
Went through the rest of the process of setting up the server.
need to set up the application key/pw pair.
Created 'Generic Application' with user/pw: platform/platform-pw
For the URL I set:http://deflaux-test.appspot.com/
Note that this is NOT https!
'Resolve IP Address' generated 127.0.0.1, which is curious...
Checked "Allow all users to authenticate."
Created a user called 'demouser' with password 'demouser-pw'
Created a group called 'demogroup'.
Create a token
C:\Users\bhoff>curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>demouser
</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica
tion-context>" -X POST http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true
* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0)
* Trying 50.17.17.19... connected
* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0)
* Server auth using Basic with user 'platform'
> POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1
> Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc=
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095
> Accept: application/xml
> Content-Type: application/xml
> Content-Length: 295
>
< HTTP/1.1 403 Forbidden
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.1
< Content-Type: text/html;charset=utf-8
< Content-Length: 1423
< Date: Wed, 09 Feb 2011 23:37:14 GMT
<
<html><head><title>Apache Tomcat/6.0.26 - Error report</title><style><!--H1
H2
H3
BODY
B
P
A
A.name
HR
--></style> </head><body><h1>HTTP Status 403 - Client with address "140.107.149.249", and hostname "140.107.149.249" is fo
rbidden from making requests to the application, platform.</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Client with address "140.107.149.249", and
hostname "140.107.149.249" is forbidden from making requests to the application, platform.</u></p><p><b>description</b> <u>Access to the specified resource (Client with address "140.107
.149.249", and hostname "140.107.149.249" is forbidden from making requests to the application, platform.) has been forbidden.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.
0.26</h3></body></html>* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact
* Closing connection #0
Added the remote address. Now:
C:\Users\bhoff>curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>demouser
</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentica
tion-context>" -X POST http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session?validate-password=true
* About to connect() to ec2-50-17-17-19.compute-1.amazonaws.com port 8095 (#0)
* Trying 50.17.17.19... connected
* Connected to ec2-50-17-17-19.compute-1.amazonaws.com (50.17.17.19) port 8095 (#0)
* Server auth using Basic with user 'platform'
> POST /crowd/rest/usermanagement/1/session?validate-password=true HTTP/1.1
> Authorization: Basic cGxhdGZvcm06cGxhdGZvcm0tcHc=
> User-Agent: curl/7.21.0 (amd64-pc-win32) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3
> Host: ec2-50-17-17-19.compute-1.amazonaws.com:8095
> Accept: application/xml
> Content-Type: application/xml
> Content-Length: 295
>
< HTTP/1.1 201 Created
< Server: Apache-Coyote/1.1
< X-Embedded-Crowd-Version: 2.1.1
< Set-Cookie: JSESSIONID=07302C98E4992295F04F0FA49BA44B05; Path=/crowd
< Cache-Control: no-cache, no-store, no-transform
< Location: http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMqaRBLAyQ00
< Content-Type: application/xml
< Content-Length: 422
< Date: Wed, 09 Feb 2011 23:40:10 GMT
<
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><session expand="user"><token>OcdmfMhMTeEUmMqaRBLAyQ00</token><user name="demouser"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazo
naws.com:8095/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/session/OcdmfMhMTeEUmMq
aRBLAyQ00"/></session>* Connection #0 to host ec2-50-17-17-19.compute-1.amazonaws.com left intact
* Closing connection #0
Now to get the details for the user:
C:\Users\bhoff>curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -k -X GET http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/us
er?username=demouser
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><user name="demouser" expand="attributes"><link rel="self" href="http://ec2-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/
user?username=demouser"/><first-name>demo</first-name><last-name>user</last-name><display-name>demo user</display-name><email>bruce.hoff@sagebase.org</email><password><link rel="edit" href="http://ec2
-50-17-17-19.compute-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/password?username=demouser"/></password><active>true</active><attributes><link rel="self" href="http://ec2-50-17-17-19.comput
e-1.amazonaws.com:8095/crowd/rest/usermanagement/1/user/attribute?username=demouser"/></attributes></user>
Now to try running from the web:
changed the version in appengine-web.xml to 'repoAndAuth' then ran
mvn gae:deploy
from projects/trunk/services
Result: 'build failure'
mvn gae:run doesn't work either
needed to run: 'mvn gae:unpack'
Now, when I 'mvn gae:run' I can test from localhost:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://localhost:8080/repo/v1/session
yeah! Next step deploy:
set deployment to version 'auth' on deflaux-test.appspot.com
ran: mvn gae:deploy
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://auth.latest.deflaux-test.appspot.com/repo/v1/session
got error, probably because Crowd isn't accepting messages from the app
ping auth.latest.deflaux-test.appspot.com
Pinging appspot.l.google.com [74.125.127.141] ...
Added 74.125.127.141 to allowed list for 'platform' in Crowd.
Still didn't work. Added more debug info, and found that this is the rejected IP: 74.125.75.17
Added the IP to the allowed list. It works!!
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST https://auth.latest.deflaux-test.appspot.com/repo/v1/session
Hitting GAE via https also works:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -k -X POST https://auth.latest.deflaux-test.appspot.com/repo/v1/session
!!!!
Now to create a Crowd server that talks SSL:
Created another micro instance
ec2-75-101-179-108.compute-1.amazonaws.com
and followed the earlier instructions to set up Crowd.
BEFORE starting it up, I followed the instructions for enabling SSL:http://confluence.atlassian.com/display/CROWD/Configuring+Crowd+to+Work+with+SSL
I tweaked the Connector element in the apache-tomcat/conf/server.xml file, to add three more attributes,
seen in the existing Connector element.
<Connector port="8443" maxHttpHeaderSize="8192" maxThreads="150"
minSpareThreads="25" maxSpareThreads="75" enableLookups="false"
disableUploadTimeout="true" acceptCount="100" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
SSLEnabled="true"
keystoreFile="/home/ec2-user/.keystore" keystorePass="changeit"
connectionTimeout="20000" disableUploadTimeout="true" useBodyEncodingForURI="true"/>
Ran
keytool -genkey -alias tomcat -keyalg RSA -sigalg MD5withRSA
which creates /home/ec2-user/.keystore
and I also copied the .keystore file to my desktop ('cause I think I'll need to stick it in my appengine application).
Edited crowd.properties: The instructions say to edit
<Crowd-Home-Directory>/crowd.properties
i.e.
/var/crowd-home/crowd.properties
but this directory doesn't exist yet. I think it's created upon start up.
An educated guess is to edit it here:
/usr/local/atlassian-crowd-2.1.1/etc/crowd.properties
with the expectation that it will ultimately be moved to /var/crowd-home.
But looking at this file I see only " (CROWD)", not the protocol specific URLs.
Made apache-tomcat/bin/setenv.sh
look like
JAVA_OPTS="-Xms128m -Xmx512m -XX:MaxPermSize=256m -Dfile.encoding=UTF-8 $JAVA_OPTS -Djavax.net.ssl.keyStore=/home/ec2-user/.keystore
-Djavax.net.ssl.keyStorePassword=changeit -Djavax.net.ssl.trustStore=/home/ec2-user/.keystore -Djavax.net.ssl.trustStorePassword=changeit"
export JAVA_OPTS
Ran /user/local/atlassian-crowd-2.1.1/start_crowd.sh
but I don't see /var/crowd-home
ps doesn't show any Java process running.
One problem is that I entered
disableUploadTimeout="true"
twice in the connector.
After removing the duplicate and running 'start_crowd.sh' again, ps shows a Java process running.
Still hasn't made /var/crowd-home
but log files don't show any errors.
Try going to https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/console
I had to go through the 'untrusted certificate' warning in Firefox, but it works.
Went through the rest of the server set up.
Note: The settings include a max number of days 'til the user has to change her password.
For now I'm not using this.
Added demouser, demouser-pw
Added application with user/pw: platform/platform-pw
Now let's try to generate a session from curl:
curl -u platform:platform-pw -H "Accept: application/xml" -H "Content-Type: application/xml" -v -k -d "<?xml version='1.0' encoding='UTF-8 '?> <authentication-context> <username>demouser</username> <password>demouser-pw</password> <validation-factors> <validation-factor> <name>remote_address</name> <value>140.107.179.234</value> </validation-factor> </validation-factors> </authentication-context>" -X POST https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session?validate-password=true
Got the error saying I have to add 140.107.149.249
After adding this address to the 'allowed' list, it works. The response is:
<?xml version="1.0" encoding="UTF-8 " standalone="yes"?><session expand="user"><token>2ffROYdzaeThREyIPbV7Wg00</token><user name="demouser"><link rel="self" href="https://ec2-75-101-179-108.compute-1.a
mazonaws.com:8443/crowd/rest/usermanagement/1/user?username=demouser"/></user><link rel="self" href="https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/1/session/2ffROYd
zaeThREyIPbV7Wg00"/></session>
Tried running Java app. It fails because I don't have the necessary certificate in Java's keystore.
The documentation seems to indicate that this isn't a problem for appengine's 'fetch' service.
Now try to hit the service, running locally:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://localhost:8080/repo/v1/session
ooops it's changed:
curl -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST http://localhost:8080/repo/v1/session
Didn't work: fetch service runs but prints message:
java.io.IOException: Could not fetch URL: https://ec2-75-101-179-108.compute-1.amazonaws.com:8443/crowd/rest/usermanagement/latest/session?validate-password=true
Could it be that my IP address has changed?
It is 140.107.177.53
Yes, that's new! I added it and reran the request:
... with the same result.
Just for fun, try deploying to a new app version called 'ssl'. The test is now:
curl -k -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST https://ssl.latest.deflaux-test.appspot.com/repo/v1/session
fails :^(
Looking at the stack trace in the GAE console, it's rejecting the IP address 74.125.66.80. Added the address in the Crowd application console and reran:
fails :^(
hey, the address changed again, this time to: 74.125.75.1
Added this address .... it worked!!
C:\Users\bhoff\eclipse projects\trunk\services\repository>curl -k -H "Content-Type:application/json" -H "Accept:application/json" -d "
" -X POST https
://ssl.latest.deflaux-test.appspot.com/repo/v1/session