Google App Engine for Python ships with the capability to manage user accounts without the need of any additional library. This functionality is, however, insufficiently documented. This post will be structured as a step-by-step tutorial addressing user registration, login, password reset and a few other details.
The webapp2 framework on Google App Engine for Python 2.7 is definitely a step forward from the original webapp.
Despite the increase in flexibility and functionality, however, there are a few items that are still more laborious than in other frameworks. The most notable aspect is user account management.
Unsurprisingly, since it is meant to run on Google’s App Engine, using Google Accounts with webapp2 takes one line of code. OpenID authentication, while still defined experimental, is almost trivial to implement as well. There are some open source projects like SimpleAuth that attempt to offer a standard and unified API to handle signing in with Google, OAuth and OpenID accounts.
While it generally makes sense to offer support for authentication through popular services – it decreases friction for new users to try a service – in some cases users may prefer having a special login to access your application.
As experience teaches us, managing passwords securely is not a trivial task, and users legitimately expect application developers to take all the necessary measures to protect their passwords.
Since this is a use case that has to be considered countless time, there is significant value in using library functions to handle user accounts.
Here is how to do that using the functionalities embedded in the webapp2_extras package that is distributed with all standard installations of App Engine for Python 2.7.
Basics and prerequisites
Our main interface when dealing with authentication is webapp2_extras.auth. This module leverages the rest of webapp2′s infrastructure to offer us a simpler way to manage user authentication.
In particular, it relies on
webapp2_extras.securityto handle password hashing (so that passwords are never stored in clear text) and random string generation;webapp2_extras.sessionsto identify requests coming from the same user as part of a client-server conversation.
Since we are implementing our own user database, we will need to define a custom model class to represent users in our application. The auth framework described here works under the assumption that this model class defines the following instance method:
get_id(self)
and the following class methods:
get_by_auth_token(cls, user_id, token)get_by_auth_password(cls, auth_id, password)create_auth_token(cls, user_id)delete_auth_token(cls, user_id, token)
The specific role of each of those methods is described here.
In addition to that, if you are interested in using the code for password reset provided as part of this tutorial, you will also need to implement the following class method:
get_by_auth_token(cls, user_id, token, subject='auth')
For each request, our application will then be able to do the following:
- read a cookie to figure out whether the current belongs to an existing session,
- if a cookie is found, read an authentication from it and load the corresponding session (if present) from the session store backend,
- loads some cached user information from the session.
We will be able to tune some details of the process via configuration options.
Extending the default User model
webapp2 already contains a reference User model for Google App Engine that uses NDB for storage. If you are willing to use that, you will find it good enough for most of the needs you may have.
In particular,
- it is an Expando model – it can include properties that were not specified as part of the class definition but are added at run-time, so it will not be a problem if your application needs to store some specific user information. New properties are indexed by default, so queries should still be fast.
- it allows you to set uniqueness constraints – while NDB does not support unique properties by default, webapp2′s User model uses a custom mechanism to allow to allow that.
Oddly enough – since the User class is clearly well designed – the reference implementation does not offer any method to update the password for a given user once it has been set. Because of that, it is a good idea to extend the User model to add a
set_password method that will have the responsibility to securely hash passwords using the security module.
You can find an example User model implementation below: it will contain the password setter and the get_by_auth_token class method (modeled after the one already provided by webapp2) that has been introduced in the previous section.
import time
import webapp2_extras.appengine.auth.models
from google.appengine.ext import ndb
from webapp2_extras import security
class User(webapp2_extras.appengine.auth.models.User):
def set_password(self, raw_password):
"""Sets the password for the current user
:param raw_password:
The raw password which will be hashed and stored
"""
self.password = security.generate_password_hash(raw_password, length=12)
@classmethod
def get_by_auth_token(cls, user_id, token, subject='auth'):
"""Returns a user object based on a user ID and token.
:param user_id:
The user_id of the requesting user.
:param token:
The token string to be verified.
:returns:
A tuple ``(User, timestamp)``, with a user object and
the token timestamp, or ``(None, None)`` if both were not found.
"""
token_key = cls.token_model.get_key(user_id, subject, token)
user_key = ndb.Key(cls, user_id)
# Use get_multi() to save a RPC call.
valid_token, user = ndb.get_multi([token_key, user_key])
if valid_token and user:
timestamp = int(time.mktime(valid_token.created.timetuple()))
return user, timestamp
return None, None
Setting up the configuration
With what we have seen so far, we are now able to configure our application. We will need to set some properties as follows:
config = {
'webapp2_extras.auth': {
'user_model': 'models.User',
'user_attributes': ['name']
},
'webapp2_extras.sessions': {
'secret_key': 'YOUR_SECRET_KEY'
}
}
In particular,
user_modelis the name of the custom User model class we described earlier,user_attributesis a list of attributes in the User model that will be cached in the session. Ideally, frequently accessed properties should be stored here. The full User model will be accessible by querying the datastore.secret_keyis the key used to secure the hash signature calculation for session cookies.
Creating a base handler class
Before writing the actual handlers that will implement the business logic to sign up and authenticate users we will group some utility functions in a base handler class, which will be extended by all the following handler classes.
This will ensure that all handlers will inherit a set of useful utility functions and properties to access user data and infrastructure classes, but also ensures that all session data is properly saved on each request (see dispatch).
class BaseHandler(webapp2.RequestHandler):
@webapp2.cached_property
def auth(self):
"""Shortcut to access the auth instance as a property."""
return auth.get_auth()
@webapp2.cached_property
def user_info(self):
"""Shortcut to access a subset of the user attributes that are stored
in the session.
The list of attributes to store in the session is specified in
config['webapp2_extras.auth']['user_attributes'].
:returns
A dictionary with most user information
"""
return self.auth.get_user_by_session()
@webapp2.cached_property
def user(self):
"""Shortcut to access the current logged in user.
Unlike user_info, it fetches information from the persistence layer and
returns an instance of the underlying model.
:returns
The instance of the user model associated to the logged in user.
"""
u = self.user_info
return self.user_model.get_by_id(u['user_id']) if u else None
@webapp2.cached_property
def user_model(self):
"""Returns the implementation of the user model.
It is consistent with config['webapp2_extras.auth']['user_model'], if set.
"""
return self.auth.store.user_model
@webapp2.cached_property
def session(self):
"""Shortcut to access the current session."""
return self.session_store.get_session(backend="datastore")
def render_template(self, view_filename, params={}):
user = self.user_info
params['user'] = user
path = os.path.join(os.path.dirname(__file__), 'views', view_filename)
self.response.out.write(template.render(path, params))
def display_message(self, message):
"""Utility function to display a template with a simple message."""
params = {
'message': message
}
self.render_template('message.html', params)
# this is needed for webapp2 sessions to work
def dispatch(self):
# Get a session store for this request.
self.session_store = sessions.get_store(request=self.request)
try:
# Dispatch the request.
webapp2.RequestHandler.dispatch(self)
finally:
# Save all sessions.
self.session_store.save_sessions(self.response)
Note – You may want redefine the view-related methods to something different if you use a different templating engine.
Registration: create new users
If we are using the model class discussed in the previous sections, in order to create a new User we just need to call the create_user method.
class SignupHandler(BaseHandler):
def get(self):
self.render_template('signup.html')
def post(self):
user_name = self.request.get('username')
email = self.request.get('email')
name = self.request.get('name')
password = self.request.get('password')
last_name = self.request.get('lastname')
unique_properties = ['email_address']
user_data = self.user_model.create_user(user_name,
unique_properties,
email_address=email, name=name, password_raw=password,
last_name=last_name, verified=False)
if not user_data[0]: #user_data is a tuple
self.display_message('Unable to create user for email %s because of \
duplicate keys %s' % (user_name, user_data[1]))
return
user = user_data[1]
user_id = user.get_id()
token = self.user_model.create_signup_token(user_id)
verification_url = self.uri_for('verification', type='v', user_id=user_id,
signup_token=token, _full=True)
msg = 'Send an email to user in order to verify their address. \
They will be able to do so by visiting <a href="{url}">{url}</a>'
self.display_message(msg.format(url=verification_url))
create_user will accept the following parameters:
- an authentication id: the token (such as a username or an email address) users will use to identify themselves when trying to access our application. While users can have multiple authentication ids, only one is allowed at creation.
- a list of unique properties (in this case,
email_address): if this is specified, webapp2 will not allow us to create new users if others share the same values for the properties in this list. - name/value pairs that are going to be set as properties of the resulting User model. If you want to add your own fields, this where to do it. If a parameter called
password_rawis present, it will be assumed to be the user password that will be used for authentication; webapp2 will hash it and store the hash the password field: we do not want to store passwords in clear text, do we?
We also create a signup token and associate it to the newly created account: we will use to confirm the email address that has been provided during registration.
Note that we are accessing the model as self.user_model rather than calling it directly, so that we are free to change which implementation to use by updating the application configuration.
Login and logout
If users are created as in the previous session, login is quite simple: the get_user_by_password method can be used to retrieve a user by their credentials. In addition to the user credentials, the method accepts some additional parameters. The one we care about (and the only one we use here) is remember: when set to True, the cookie used to identify the session is saved as persistent and the browser will keep it even after the user will close its window.
class LoginHandler(BaseHandler):
def get(self):
self._serve_page()
def post(self):
username = self.request.get('username')
password = self.request.get('password')
try:
u = self.auth.get_user_by_password(username, password, remember=True)
self.redirect(self.uri_for('home'))
except (InvalidAuthIdError, InvalidPasswordError) as e:
logging.info('Login failed for user %s because of %s', username, type(e))
self._serve_page(True)
def _serve_page(self, failed=False):
username = self.request.get('username')
params = {
'username': username,
'failed': failed
}
self.render_template('login.html', params)
The implementation above renders the login form when the request comes via GET and processes the credentials upon POST. When authentication fails it renders the login form and passes the username to the template so that the corresponding field can be pre-filled.
Implementing logout is even simpler: it is sufficient to get rid of the user session.
class LogoutHandler(BaseHandler):
def get(self):
self.auth.unset_session()
self.redirect(self.uri_for('home'))
Email confirmation and password reset
Signup tokens are one of the undocumented features of webapp2 (and they may possibly subject to change), but they can be quite handy when implementing a flow to confirming email addresses or recover passwords.
As mentioned before, the webapp2 uses authentication tokens to identify users after they logged in: they are meant to be securely shared by a client and the server and exchanged when a client needs to prove its identity.
As you probably imagine, this mechanism can be generalized to handle email confirmations and password resets: when websites send us an activation link after a registration, the URL usually contain their equivalent of signup tokens.
webapp2 sets a subject property for each of the tokens it generates, so the only difference between auth token and signup token is the value for that property. So, why do we want to use signup tokens?
Setting a different value for that property allows us to partition tokens by their purpose: we can then implement useful features as deleting all the password reset tokens that have not been used in 48 hours.
Here is a sample verification handler that is able to process email verification links:
class VerificationHandler(BaseHandler):
def get(self, *args, **kwargs):
user = None
user_id = kwargs['user_id']
signup_token = kwargs['signup_token']
verification_type = kwargs['type']
# it should be something more concise like
# self.auth.get_user_by_token(user_id, signup_token
# unfortunately the auth interface does not (yet) allow to manipulate
# signup tokens concisely
user, ts = self.user_model.get_by_auth_token(int(user_id), signup_token,
'signup')
if not user:
logging.info('Could not find any user with id "%s" signup token "%s"',
user_id, signup_token)
self.abort(404)
# store user data in the session
self.auth.set_session(self.auth.store.user_to_dict(user), remember=True)
if verification_type == 'v':
# remove signup token, we don't want users to come back with an old link
self.user_model.delete_signup_token(user.get_id(), signup_token)
if not user.verified:
user.verified = True
user.put()
self.display_message('User email address has been verified.')
return
elif verification_type == 'p':
# supply user to the page
params = {
'user': user,
'token': signup_token
}
self.render_template('resetpassword.html', params)
else:
logging.info('verification type not supported')
self.abort(404)
This handler is meant to be used with a route using a template that matches URLs like /v/USERID-TOKEN. You can configure it as follows (please refer to the sample code at the end of this article for the full routes configuration):
webapp2.Route('/<type:v|p>/<user_id:\d+>-<signup_token:.+>',
handler=VerificationHandler, name='verification')
Two minor notes on this item:
- For increased security, we may require users to enter their password before authenticating them.
- Ideally, we may want to use a different subject for email confirmation and password reset tokens.
Ensure users are logged in
Now that everything else is in place, we can decide whether users are allowed to access certain resources depending on their logged in state.
The following decorator can be used to annotate handler methods that require users to be logged in.
def user_required(handler):
"""
Decorator that checks if there's a user associated with the current session.
Will also fail if there's no session present.
"""
def check_login(self, *args, **kwargs):
auth = self.auth
if not auth.get_user_by_session():
self.redirect(self.uri_for('login'), abort=True)
else:
return handler(self, *args, **kwargs)
return check_login
Just placing @user_required, as in the following example, before those methods will ensure that anonymous users will be directed to a login page when attempting to go through the annotated handler method.
class AuthenticatedHandler(BaseHandler):
@user_required
def get(self):
self.render_template('authenticated.html')
Finishing touches
Before actually using this in code in production, there is at least one task we should take care of: calls that send passwords (like login, signup, password reset) should be using https.
This is quite easy to do and the documentation is quite straightforward: just follow the instructions here and you will be all set.
Your app.yaml file should include the following once you are done:
handlers: - url: .* script: main.app - url: /signup script: main.app secure: always - url: /login script: main.app secure: always - url: /forgot script: main.app secure: always libraries: - name: webapp2 version: "2.5.1"
Of course, we will also need some views to be able to use this application. While this is the typical task that is left as an exercise to the reader, the example implementation you will find in the following section will contain a fully working application you can play with.
Reference code
You can find a ready to use application skeleton on GitHub. Feel free to play with it to experiment the full flow described in this post, use it to bootstrap your project and to improve on it. Just post a comment if you have any question or suggestion.
If you found this post useful you should follow me on twitter.
Thank you!!! Just what I was looking for!
Glad it helped out, just ask if you have any trouble!
hi! I’m new with GAE and I created a app with webapp2 authentication but when a update it to production don’t work and I don’t have any idea what the problem, I think that a need to setup something in my acount but I’m not sure
Thank you very much. Extremely helpful for beginners like me.
Thank you just what I needed… Really thanks
Many thanks. Very helpful post.
Maybe you want to mention to first-time users of webapp2 to add it to the libraries in app.yaml, for example:
libraries:
- name: webapp2
version: 2.5.1
Thanks for sharing!
Thank you all for your feedback.
anihatzis, your suggestion does indeed make sense: I just updated the code snippet to reflect that.
Thanks
Great article. It may help people if you provide a snippet for your BaseHandler class.
Thanks, definitely good advice. Just updated the post to include that information too.
My Mistake – You have BaseHandler in your GitHub code.
One thing I’m considering is setting auth_fail_action: unauthorized
in app.yaml. Then setting a custom error handler.
app.error_handlers[401] = handle_401
One other thing – on the new dev server by default I found the link to browse users should be “http://localhost:8000/datastore?kind=User”
Has anyone had an issue with this where login/logout actions won’t take effect until either python is restarted, a file is edited, or the dev server is restarted? Seems to be some sort of caching that I can’t find.
Any help would be appreciated, thanks.
I figured out my problem with the values not changing, it was in my own code in the custom render function I had made.
Thanks for this write-up, it is a huge help!
Thank you all for the comments.
Robert, I did remove the link to browse users, as there is no reliable way to get that information without hardcoding it.
Also, if you actually perform the app config change you mentioned might consider sending a pull request with that.
I’m not using HTTPS, (apart from for when they enter their password).
That means tokens can be stolen. A hacker will be able to access most of the website and most of a users account, however to perform important operations, they will be redirected to https://*.appspot.com and need to re-enter their password.
Not being secure isn’t ideal but saves 9$ a month on SSL certificates – what do you think?
@Robert: Depends on what kind of app you have. Basically, if it comes to user information, I would try to do anything reasonable to prevent harm.
Everyone working with user management and sensitive user information may want to read some good advice here:
http://stackoverflow.com/questions/549/the-definitive-guide-to-forms-based-website-authentication?rq=1
The webapp2 implementation of authentication, even the one based on username and password, relies on cookie storing session. this session is either a store itself or a key to the server-side store (depeding on the session “backend” option).
So auth_token is never passed to the client and client is not expected to pass it back to be able to access resources.
What if I was wringing an API based service, similar, for example, to app.net (ADN).
The clients to my API are not browsers – but mobile apps.
My understanding is that if I used webapp2 auth, the client apps would have to simulate browser, that is storing secure cookie as if it was a token.
ADN authentication is based on returning token explicityly in the body of the response, and having client submit the token either in Bearer header, query string or body of the POST request. I would love to do the same. The implementation however would not use any of the goodness bestowed upon us by webapp2.
My questions
1. Did I describe the situation correctly?
2. What would you do if you were writing API only servce?
Hi Michael,
yes, you did describe the situation perfectly.
I believe Google Cloud Endpoints would be the perfect solution for your need as they would handle both exposing an API on the server side and generating code for the client side. Endpoints support authenticated calls by leveraging OAuth, which would be compatible with your needs.
On the other hand, as the documentation states,
So if you go with that, you might need to make some changes on your app while the API evolves.
A more stable alternative (at least in the short term) that would require a bit more work would be to implement an OAuth provider in your server application (perhaps using an existing library like appengine_oauth_provider).
Does this answer your question?
Thanks Alessandro
I would like to use GAE. I would love to have an example. Do you have a simple app like that? There is gae-boilerplate but they use session. I shall check out your suggestions.
appengine_oath_provider seems to require django. I am using webapp2. Not sure how to go about adapting.
Unfortunately I do not have any example available, but I think the library I had linked above does not depend on Django: it is based on one that used to be, but the maintainer had replaced the django calls with the GAE equivalents.
Another alternative worth checking out is this overview of OAuth for Python.
I have managed somehow to make it work with webapp2 User, Auth and Session.
Session store is still used to keep token. Session cookie is ignored, I don’t know how to avoid it being sent. Token is extracted from the header. Once this project’s rush time is over in about one month I hope to distill what I did into a clear example
Hi Alessandro,
First, thank you very much for this great work…
It would be great if you can create an “admin” page where you can see all the accounts and the ability to remove them if necessary.
Again, congratulations, and I hope you will consider my comments.
Regards,
Vincent
PS : Sorry for my english
Great article, thanks.
In my application time to time users can not login immediately after registration.
I’m calling self.auth.get_user_by_password(email, password) after model.User.create_user(email, password_raw=password).
Time to time I got InvalidAuthIdError.
You’ve never experienced this?
Thank you folks,
Michael, I am not sure you can prevent the cookie from being sent, once you have it. If you’ll share the example once you have it, I might try having a look.
Vincent, I did not create an admin view because I tried to keep this project a skeleton that people could build upon. If you don’t believe AppEngine datastore admin is enough for your purposes and want more, you can fork the project and contribute an admin section.
Dmitry, I have never experienced that issue: can you find anything relevant in the logs?
First of all thanks for this great article. I’ve followed it and also changed some bits to fit my page.
I’ve run into a problem when I try to let users change its properties (name, surnames,…).
My problem is that user_info is a cached property so, when I update a user values, old values keep showing on my page. I can check through the admin console that the values have changed on db, but the values on user_info are the old ones.
I’ve tried several things (like removing user_info to force to be calculated again), but the problem remains. I guess that user_info is calculated using another cached property which stills use old values.
Any tip on solving this issue?
Thanks in advance
Has anyone tried hooking this up with cloud endpoints? I want to use cloud endpoints for part of my API.
This looks promising: http://stackoverflow.com/questions/15690831/cloud-endpoints-http-cookies#15693618
hi has anyone being able to query all users from the datastore? I want to check for information of User (SELECT * FROM User) i had an error of Kind not found User
I’ve found the solution to my problem. I guess I wasn’t on my best time.
If anybody has problems updating user data on a session, here is how I solved it. Having updated user data on ‘data’:
currdata = self.auth.get_session_data(pop=True)
currdata.update(data)
self.auth.set_session(currdata, token=currdata['token'], token_ts=currdata['token_ts'],
cache_ts=currdata['cache_ts'], remember=currdata['remember'])
@Kelvin, I guess that if you have this kind of error is because you have not created (and put) any instance of User
Great job Otger!
Kelvin, it would be a lot easier to help you if you could share a snippet of the code you are using. Maybe you can try posting the question on StackOverflow or pasting it in a gist?
Robert, I have been playing with Endpoints and they are definitely interesting. One thing you may want to explore, however, is that if you rely on cookies for authentication, you may have trouble if you try to authenticate clients that do not run in the browser (e.g. mobile apps)