RESTing securely

Friends don't let friends write insecure code

Silvrback blog image

Django is a powerful python web framework, which follows the DRY (Don’t repeat yourself), and batteries included philosophies. At @Kipinhall, we are proud to use Django because of the rich ecosystem it offers via plugins & its ever expanding community.

Recently we created a provisioning API for 3rd party apps to access our course data, add/update users, create new classes/schedules etc. The API follows the standard REST protocol, which can be cumbersome to roll out on your own. Luckily, Django provides an excellent plugin called Django Tastypie, which does lotta heavy lifting in writing REST compliant API's [Also checkout Django Rest Framework]. Furthermore, Tastypie comes with excellent documentation and its integration into the project was a breeze. More @ tastypie.

This post will highlight some of our security requirements of the API and customizations we did to Tastypie to meet those requirements.

Authentication

Problem

Tastypie comes with some default authentications, which don't necessarily fit our requirements. Our needs are similar to the way Google Analytics or Mixpanel handle their client app requests, where each client app owns its own data.

The following illustrates the use case,

  • A 3rd party app will register with Kipinhall and get an API Key.
  • They will use this API Key to perform actions such as adding users, authenticating users, creating content on their behalf, some administrative stuff etc.
  • The data is fully owned by the app and cannot be viewed or modified by other 3rd party app.

Because Tastypie treats all users equally, there is no clear way to distinguish between a 3rd party or an Application User. In addition to that, it doesn't fully cover our security needs. To illustrate a few,

  • API Key for web based app is clearly exposed in the JS scripts. Anyone can then use that and make a call on behalf of the user.
  • No domain restriction on the API calls.
  • Token based authentication is directly tied to a user and does not expire.
  • It needs to support chained authentication, which is only successful when all of the authentications are successful.
  • Just like request.user, we need access to the client information. We can then filter the data, add analytics based on the usage & much more.

I can keep on going, but you get the idea. Fortunately, Tastypie is fully customizable and you can plugin your own authentication. So lets code.

Solution

Client

To segregate between 3rd party and an Application User, we will create a new model called Client in a package say third_party.

class Client(models.Model):
  name = models.CharField(_('Name'), max_length = 25)
  base_uri = models.URLField(_('Base URI'))
  api_key = models.CharField(_('API Key'), max_length = 25)
  secret_key = models.CharField(_('Secret Key'), max_length = 25)
  allowed_domain = models.TextField(_('Allowed Domain'))
  allow_subdomains = models.BooleanField(_('Allow Subdomains'))
  throttle_threshold = models.PositiveIntegerField(_('Throttle Threshold'), 
                                                      default = 100)

  def save(self, force_insert = False, force_update = False, using = None):
    """
    Note `api_generator, sec_key_generator` is our custom methods, 
      but for starters you can use `get_random_string` 
      from django's crypto package.
    """
    if self.pk is None:
      self.api_key = api_generator(25)
      self.secret_key = sec_key_generator(25, settings.SALT, time.time())

    super(Client, self).save(force_insert = force_insert, 
        force_update = force_update, using = using)

The model is pretty explanatory. Here are some salient points,

  • We have a self serving 3rd party app portal, where the user can request the API key.
  • We hide the api_key and secret_key fields on the form and generate them on a new object creation.
  • We store the domain name in allowed_domain field
  • allow_subdomains flag will allow subdomains e.g. *.domain.com.
  • On a successful creation, we show the user the newly created api_key and the secret_key, which they will use to sign their requests.

User Token Header

The 3rd party client will initiate the first request by authenticating the user via Oauth or simple user/password mechanism.

On successful authentication, we return the user token in the response headers, which the client is expected to store locally. This can be done via simple django view that returns the desired information.

On subsequent requests, client is expected to pass HTTP_TOKEN header with the username and the token to ensure the right user is connected to the request.

TOKEN: <username>:<token>

Make sure you add tastypie to INSTALLED_APPS and sync your DB. On every user creation, it creates a corresponding entry in the ApiKey table.

Tastypie comes with an existing token based user authentication, namely ApiKeyAuthentication, which we will slightly modify to look for TOKEN instead of the typical Authorization header it expects.

Do not be confused by the name. The user related api key is really a token. Furthermore, we modified ApiKey to hold expires field, which ensures tokens don't last forever.

class TokenUserAuthentication(ApiKeyAuthentication):
    """
    Handles token auth, in which a client provides a username & token,
    which translates to `key` in the ApiKey table.

    Note we are inheriting from `ApiKeyAuthentication`, which takes care
    of token lookup. We are only overriding the method
    `extra_credentials`.

    Since the "Authorization" is now reserved for client header, we want 
    to use a different header for the user. Token is industry standard for 
    user based authentication, so we will stick with that. 
    Also, Api Key is industry norm for issuing it to 3rd party apps.

    Not shown in here is validating token expiry date but its fairly easy by 
    adding `expires` date-time field to `ApiKey`. In addition to 
    that, we expect our clients to sign the token with the `secret_key` 
    but I left it out for the sake of brevity.
    """

    def extract_credentials(self, request):
        if request.META.get('HTTP_TOKEN'):
            username, token = request.META['HTTP_TOKEN'].split(':', 1)
            return username, token
        else:
            raise ValueError('Incorrect token header.')

Client Header

To ensure no one can impersonate the client, we expect the 3rd party apps to send an extra header identifying their identity. If you recall, the client requested for an API key and got in return an api_key and secret_key. Those two keys will be used in signing the header.

Note, its recommended that the client performs these operations on the backend since the secret_key should never be exposed on the website. Fortunately, its lot easier for devices since they aren't exposed the same way the website pages are.

Signature

The signing process requires the client to compute the request signature by signing the canonicalized URL.

Assuming the 3rd party is using django, they can sign the request as follows

import hashlib
import base64

def get_signed_request(request):
  #prep string
  hashable_string = b'%s%s%s' % (client.api_key, 
           client.secret_key, 
           request.build_absolute_uri(request.get_full_path()))

  m = hashlib.sha256(hashable_string)
  return base64.b64encode(m.hexdigest())

The client will dump the above value in every request as a AUTHORIZATION header, along with the api_key.

#format
Authorization: Kipin <api_key>:<signature>
#assume api_key is 123456789

Authorization: Kipin 123456789:YTI5YmFjNzIzY2EyZDU5ZWQ3OGEyZDcxNWUxN2U5MmY=

Client Authentication

Now that we have the Authorization header in the request, we need to verify its signature and depending on the result, we will either allow the request or raise a 401 unauthorized request error.

Tastypie provides multi authentication mechanism, where it checks list of authentication protocols till at least one of them returns true.
Since we want all of the authentication protocols to succeed, we will slightly modify to ensure it does exactly that.

class ChainedAuthentication(MultiAuthentication):
    """
    Chained authentication ensures all authentications are passed 
    and return True else it will return False. 
    """
    def is_authenticated(self, request, **kwargs):
    """
    Ensures all authentications return True. Should return either ``True`` 
        if allowed, ``False`` if not or an ``HttpResponse`` 
        if you need something custom.
    """
    authorized = False

    for backend in self.backends:
        check = backend.is_authenticated(request, **kwargs)

        if check:
            if isinstance(check, HttpUnauthorized):
                return check
            else:
                request._authentication_backend = backend
                authorized = check == True

    return authorized

Now is a good time to write the client authentication class that will verify the client to who they claim to be.

class ClientAuthentication(Authentication):
    """
    Verifies the client signature

    """

    def _unauthorized(self):
        return HttpUnauthorized()

    def extract_credentials(self, request):
        """
        Just extract the above header by splitting the 
        value by ':' as API Key and signature.
        """
        if request.META.get('HTTP_AUTHORIZATION') and     
            request.META['HTTP_AUTHORIZATION']
                               .lower().startswith('kipin '):
            auth_type, data = request.META['HTTP_AUTHORIZATION'].split()

            if auth_type.lower() != 'kipin':
                raise ValueError("Incorrect authorization header.")

            return data.split(':', 1)

        raise ValueError("Incorrect authorization header.")

    def is_authenticated(self, request, **kwargs):
        """
        Finds the client and checks the signature.
        """

        try:
            api_key, signature = self.extract_credentials(request)
        except ValueError:
            return self._unauthorized()

        if not api_key or not signature:
            return self._unauthorized()

        #first check the client
        try:
            lookup_kwargs = {api_key: api_key}
            client = Client.objects.get(**lookup_kwargs)
        except (Client.DoesNotExist, Client.MultipleObjectsReturned):
            return self._unauthorized()

        #verify_signature is exactly the same code as get_signed_request 
        #except that it accepts list of strings to sign 
        #and compares against the signature passed.
        #`verify_signature(self, signature, *args)`
        if not client.verify_signature(base64.b64decode(signature), 
            request.build_absolute_uri(request.get_full_path())):
            return False

        #set the client, so views/resources can access this
        request.client = client

        return True

Restricting Domains

There are two parts to domain validation,

  • One is enabling CORS on the server so 3rd party apps can make a cross-domain request. We will add the value from the field Client.allowed_domains to the response on a CORS pre-flight request.
  • Check the domain origin to ensure that they match to the allowed domains for the given client. I will leave out for the reader to figure this out. [hint middleware]

Note, anyone can mimic the Origin by doing curl -H "Origin: http://allowed-domain.com", so just doing straight up validation of domain is not enough.

Lets start with adding CORS information to the response via middleware

class CorsPreFlightCheckMiddleware(object):

    """
    Pre flight check for CORS request. You can also extend this to
    add domain validation.
    """

    def process_request(self, request):
        '''
        If CORS preflight header, then create an empty body response 
       (200 OK) and return it

       Django won't bother calling any other request view/exception 
       middleware along with the requested view; it will call 
       any response middlewares.
        '''
        if (request.method == 'OPTIONS' and
            'HTTP_ACCESS_CONTROL_REQUEST_METHOD' in request.META):
            return http.HttpResponse()

        return None

    def process_response(self, request, response):
        '''
        Add the respective CORS headers for pre-flight check
        '''
        #only do this in case of 'OPTIONS' request.
        if (request.method == 'OPTIONS' and
            'HTTP_ACCESS_CONTROL_REQUEST_METHOD' in request.META):

            #default entries
            response['Access-Control-Allow-Headers'] = 'Content-Type'

            #extract the client API Key
            try:
                api_key, signature = self.extract_credentials(request)
                #verify the signature as above

                client = Client.objects.get(api_key = api_key)
                response['Access-Control-Allow-Origin'] = 
                    client.allowed_domain
                #you can store the information as a JSONField 
                #in Client model                
                response['<Other-CORS-RELATED-Headers>'] = '<value>'

            except ValueError, ObjectDoesNotExist:
                #assumes you configured this, which you should
                response['Access-Control-Allow-Origin'] = 
                    Site.objects.get_current().domain

        return response

Securing Resources

We need to secure our resources with the above authentication mechanisms.

#Since we do not believe in redundant code, lets create a base meta class.
class ClientAuthenticationMixin(object):
    authentication = ChainedAuthentication(ClientAuthentication(), 
                                           TokenUserAuthentication())

class CourseResource(ModelResource):
    """ 
    Note the `CacheableMixin` which caches our endpoints.
    """
    class Meta(ClientAuthenticationMixin, DomainValidatorMixin,
                                   ThrottleMixin, CacheableMixin):
        allowed_methods = ['get']
        queryset = Course.objects.all()    

Finally, add CorsPreFlightCheckMiddleware middleware to your MIDDLEWARE_CLASSES setting in your settings.py.

Conclusion

I skipped a lot of things, especially validating domains on an actual HTTP request, throttling based on client settings, munging sensitive information etc. So if some of the code doesn't make sense, please let me know.

You don't have to use all of the techniques above but you can start with enforcing your API users to use TLS/SSL protocol.

Note that security is ever evolving and in no way the above suggestions should be considered as an iron clad solution.

Security is all about opportunity cost to a malicious hacker. Making it tougher and longer for them to hack is a good deterrent.

follow me on @agileseeker

Saikiran YerramEverything is going to be 200 OK

comments powered by Disqus