" REST API Development with Node.js 🔙 Quay lại trang tải sách pdf ebook REST API Development with Node.js Ebooks Nhóm Zalo REST API Development with Node.js Manage and Understand the Full Capabilities of Successful REST Development — Second Edition — Fernando Doglio REST API Development with Node.js Manage and Understand the Full Capabilities of Successful REST Development Second Edition Fernando Doglio REST API Development with Node.js Fernando Doglio La Paz, Canelones, Uruguay ISBN-13 (pbk): 978-1-4842-3714-4 ISBN-13 (electronic): 978-1-4842-3715-1 https://doi.org/10.1007/978-1-4842-3715-1 Library of Congress Control Number: 2018950838 Copyright © 2018 by Fernando Doglio This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director, Apress Media LLC: Welmoed Spahr Acquisitions Editor: Louise Corrigan Development Editor: James Markham Coordinating Editor: Nancy Chen Cover designed by eStudioCalamar Cover image designed by Freepik (www.freepik.com) Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail rights@apress.com, or visit http://www.apress.com/ rights-permissions. Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the book’s product page, located at www.apress.com/9781484237144. For more detailed information, please visit http://www.apress.com/source-code. Printed on acid-free paper To my loving wife, without whom this book would’ve never happened and to my beautiful boys, without whom this book would’ve happened a lot sooner… Thank you! Table of Contents About the Author ����������������������������������������������������������������������������������������������������� xi About the Technical Reviewer ������������������������������������������������������������������������������� xiii Acknowledgments���������������������������������������������������������������������������������������������������xv Introduction�����������������������������������������������������������������������������������������������������������xvii Chapter 1: REST 101������������������������������������������������������������������������������������������������� 1 Where Did It All Start?������������������������������������������������������������������������������������������������������������������� 2 REST Constraints �������������������������������������������������������������������������������������������������������������������������� 4 Client–Server��������������������������������������������������������������������������������������������������������������������������� 4 Stateless���������������������������������������������������������������������������������������������������������������������������������� 5 Cacheable�������������������������������������������������������������������������������������������������������������������������������� 6 Uniform Interface��������������������������������������������������������������������������������������������������������������������� 7 Layered System����������������������������������������������������������������������������������������������������������������������� 9 Code-on-Demand ������������������������������������������������������������������������������������������������������������������ 10 Resources, Resources, Resources ���������������������������������������������������������������������������������������������� 11 Representations �������������������������������������������������������������������������������������������������������������������� 11 Resource Identifier ���������������������������������������������������������������������������������������������������������������� 14 Actions����������������������������������������������������������������������������������������������������������������������������������� 15 Hypermedia in the Response and Main Entry Point �������������������������������������������������������������� 20 Status Codes������������������������������������������������������������������������������������������������������������������������������� 26 REST vs. the Past������������������������������������������������������������������������������������������������������������������������ 28 Summary������������������������������������������������������������������������������������������������������������������������������������� 37 v Table of Contents Chapter 2: API Design Best Practices��������������������������������������������������������������������� 39 What Defines a Good API?����������������������������������������������������������������������������������������������������������� 39 Developer-Friendly���������������������������������������������������������������������������������������������������������������������� 40 Communication’s Protocol����������������������������������������������������������������������������������������������������� 40 Easy-to-Remember Access Points ���������������������������������������������������������������������������������������� 41 Uniform Interface������������������������������������������������������������������������������������������������������������������� 42 Extensibility��������������������������������������������������������������������������������������������������������������������������������� 46 How Is Extensibility Managed?���������������������������������������������������������������������������������������������� 46 Up-to-Date Documentation ��������������������������������������������������������������������������������������������������������� 50 Proper Error Handling ����������������������������������������������������������������������������������������������������������������� 53 Phase 1: Development of the Client��������������������������������������������������������������������������������������� 53 Phase 2: The Client Is Implemented and Being Used by End Users��������������������������������������� 55 Multiple SDK/Libraries ���������������������������������������������������������������������������������������������������������������� 56 Security��������������������������������������������������������������������������������������������������������������������������������������� 57 Accessing the System ����������������������������������������������������������������������������������������������������������� 58 Scalability ����������������������������������������������������������������������������������������������������������������������������������� 66 Summary������������������������������������������������������������������������������������������������������������������������������������� 70 Chapter 3: Node.js and REST ���������������������������������������������������������������������������������� 71 Asynchronous Programming������������������������������������������������������������������������������������������������������� 72 Async Advanced �������������������������������������������������������������������������������������������������������������������� 76 Asynchronous I/O������������������������������������������������������������������������������������������������������������������������ 81 Async I/O vs. Sync I/O������������������������������������������������������������������������������������������������������������ 85 Simplicity ������������������������������������������������������������������������������������������������������������������������������������ 87 Dynamic Typing ��������������������������������������������������������������������������������������������������������������������� 88 Object-Oriented Programming Simplified������������������������������������������������������������������������������ 89 The new Class construct from ES6���������������������������������������������������������������������������������������� 91 Functional Programming Support������������������������������������������������������������������������������������������ 93 Duck Typing ��������������������������������������������������������������������������������������������������������������������������� 94 Native Support for JSON�������������������������������������������������������������������������������������������������������� 95 vi Table of Contents npm: The Node Package Manager ���������������������������������������������������������������������������������������������� 96 Who’s Using Node.js?������������������������������������������������������������������������������������������������������������������ 98 Summary������������������������������������������������������������������������������������������������������������������������������������� 99 Chapter 4: Architecting a REST API ���������������������������������������������������������������������� 101 The Request Handler, the Pre-Process Chain, and the Routes Handler ������������������������������������ 102 MVC: a.k.a. Model–View–Controller������������������������������������������������������������������������������������������ 107 Alternatives to MVC ������������������������������������������������������������������������������������������������������������� 112 Response Handler��������������������������������������������������������������������������������������������������������������������� 116 Summary����������������������������������������������������������������������������������������������������������������������������������� 119 Chapter 5: Working with Modules ������������������������������������������������������������������������ 121 Our Alternatives ������������������������������������������������������������������������������������������������������������������������ 122 Request/Response Handling ����������������������������������������������������������������������������������������������� 122 Routes Handling ������������������������������������������������������������������������������������������������������������������ 122 Middleware�������������������������������������������������������������������������������������������������������������������������� 123 Up-to-Date Documentation�������������������������������������������������������������������������������������������������� 125 Hypermedia on the Response���������������������������������������������������������������������������������������������� 125 Response and Request Validation ��������������������������������������������������������������������������������������� 125 The List of Modules ������������������������������������������������������������������������������������������������������������� 125 Summary����������������������������������������������������������������������������������������������������������������������������������� 172 Chapter 6: Planning Your REST API ���������������������������������������������������������������������� 173 The Problem������������������������������������������������������������������������������������������������������������������������������ 173 The Specifications ��������������������������������������������������������������������������������������������������������������� 177 Choosing the Right Modules for the Job ����������������������������������������������������������������������������� 188 Summary����������������������������������������������������������������������������������������������������������������������������������� 189 Chapter 7: Developing Your REST API������������������������������������������������������������������� 191 Minor Changes to the Plan�������������������������������������������������������������������������������������������������������� 192 Simplification of the Store–Employee Relationship������������������������������������������������������������� 192 Adding Swagger UI�������������������������������������������������������������������������������������������������������������� 192 Simplified Security �������������������������������������������������������������������������������������������������������������� 193 vii Table of Contents A Small Backdoor for Swagger�������������������������������������������������������������������������������������������� 193 MVC ������������������������������������������������������������������������������������������������������������������������������������� 194 Folder Structure ������������������������������������������������������������������������������������������������������������������������ 194 The Source Code����������������������������������������������������������������������������������������������������������������������� 196 config����������������������������������������������������������������������������������������������������������������������������������� 197 Controllers ��������������������������������������������������������������������������������������������������������������������������� 197 lib����������������������������������������������������������������������������������������������������������������������������������������� 228 Models��������������������������������������������������������������������������������������������������������������������������������� 237 request_schemas���������������������������������������������������������������������������������������������������������������� 243 schemas ������������������������������������������������������������������������������������������������������������������������������ 245 swagger-ui��������������������������������������������������������������������������������������������������������������������������� 254 Root Folder�������������������������������������������������������������������������������������������������������������������������� 255 Summary����������������������������������������������������������������������������������������������������������������������������������� 259 Chapter 8: Testing your API ���������������������������������������������������������������������������������� 261 Testing 101�������������������������������������������������������������������������������������������������������������������������������� 261 The Definition ���������������������������������������������������������������������������������������������������������������������� 261 The Tools������������������������������������������������������������������������������������������������������������������������������ 264 Best Practices���������������������������������������������������������������������������������������������������������������������� 271 Testing with Node.js������������������������������������������������������������������������������������������������������������������ 272 Testing Without Modules ����������������������������������������������������������������������������������������������������� 272 Mocha���������������������������������������������������������������������������������������������������������������������������������� 275 Summary����������������������������������������������������������������������������������������������������������������������������������� 282 Chapter 9: Deploying into Production������������������������������������������������������������������� 283 Different Environments ������������������������������������������������������������������������������������������������������������� 283 The Classical Development Workflow���������������������������������������������������������������������������������� 283 Tips for Your Production Environment ��������������������������������������������������������������������������������� 286 viii Table of Contents Doing the Actual Deployment���������������������������������������������������������������������������������������������������� 291 Shipit ����������������������������������������������������������������������������������������������������������������������������������� 292 What about Continuous Integration? ����������������������������������������������������������������������������������� 295 PM2 ������������������������������������������������������������������������������������������������������������������������������������� 296 Summary����������������������������������������������������������������������������������������������������������������������������������� 302 Chapter 10: Troubleshooting��������������������������������������������������������������������������������� 303 Asynchronous Programming����������������������������������������������������������������������������������������������������� 303 The Controllers Action’s Code���������������������������������������������������������������������������������������������� 304 The Middleware Functions �������������������������������������������������������������������������������������������������� 306 Issues Configuring the Swagger UI������������������������������������������������������������������������������������������� 308 CORS: a.k.a. Cross-Origin Resource Sharing ���������������������������������������������������������������������������� 310 Summary����������������������������������������������������������������������������������������������������������������������������������� 314 Index��������������������������������������������������������������������������������������������������������������������� 315 ix About the Author Fernando Doglio has worked as a developer for the past 13 years. In that time, he has come to love the Web and has had the opportunity to work with most leading technologies, such as PHP, Ruby on Rails, MySQL, Node.js, Angular.js, AJAX, REST APIs, and others. For the past 4 years Fernando has also been working as a Technical Manager and Technical Lead for BigData projects. In his spare time, Fernando likes to tinker, learn new things, and write technical articles and books such as this one. He’s also a big open source supporter, always trying to bring new people into that realm. When not programming, he is spending time with his family. Fernando can be contacted on Twitter @deleteman123 or online at www.fernandodoglio.com. xi About the Technical Reviewer Takashi Mukoda is an international student at Purchase College/State University of New York. Now, he is taking a semester off and is back home in Japan. At Purchase College, he majors in Mathematics/Computer Science and New Media and has worked as a teaching assistant for computing and mathematics courses. Takashi likes playing the keyboard and going on hikes in mountains to take pictures. His interest in programming and art incites him to create multi-media art pieces. Some of them are built with Processing and interact with human motion and sounds. (Website: http://www.takashimukoda.com) xiii Acknowledgments I’d like to thank the amazing technical reviewer involved in the project, Takashi Mukoda, whose great feedback was a crucial contribution to the making of this book. I’d also like to thank the rest of the Apress editorial team, whose guidance helped me through the process of writing this, my first book. xv Introduction These days, everyone is finding a new way to interconnect systems; the Internet of Things (IoT), for instance, is the new kid on the block, but who knows what will come later. The point is that in order to interconnect systems, as an architect, you’re better off using standard methods that allow for a faster adoption of your technology. In particular, APIs allow for the creation of standards and can work under known and well-tested core technologies like HTTP. If you add to that a well-defined style guide like REST, you’ve got yourself the means to create a scalable, technology-agnostic, and uniform interface for your services to be consumed by your clients. Welcome to REST API Development with Node.js. This book will cover REST, API development, and, finally, how these two mix up with Node.js. Starting from a theoretical point of view, you’ll learn how REST came to be, who created it, and its characteristics. Later, you’ll move toward the practical side by going over API development and the lessons that years of experience from the community have taught us. Finally, you’ll move into a fully practical approach, and you’ll see how Node.js and its modules can help create a RESTful API. You’ll also get a taste of what a real-world development flow would be like and what it would take to both test and deploy your code into a production environment. The final chapters will be 100% practical, going over a real-world example of a RESTful API developed in Node.js. I will cover everything from the requirement gathering process, to tools selection, through actual development, and, finally, you’ll land in troubleshooting-land, where I’ll discuss the different things that can go wrong and how to tackle them. Now sit back, relax, and enjoy the reading. xvii CHAPTER 1 REST 101 Nowadays, the acronym REST has become a buzzword, and as such, it’s being thrown into the digital wind very carelessly by a lot of tech people without fully understanding what it really means. Just because you can interact with a system using HTTP, and send JSON back and forth, doesn’t mean it’s a RESTful system. REST is a lot more than that— and that is what we’ll cover in this chapter. Let’s start where it all began, with Roy Fielding’s paper, going over the main characteristics of his idea. I’ll try to explain the main aspects of it, the constraints he added, and why he added them. I’ll go over some examples and then jump backward into the past, because even though REST has proven to be a huge jump forward regarding distributed systems interconnection, before Fielding’s paper became popular, developers were still looking for solutions to the problem: how to easily interconnect a nonhomogeneous set of systems. I’ll do a quick review of the options developers had back then to interconnect systems, mainly going over SOAP and XML-RPC (the two main players before REST). In the end, I’ll jump back to our current time, comparing the advantages that REST brought us and thus showing why is it so popular today. But first, a small clarification is in order: As you’ll read in just a few minutes, REST is protocol-independent (as long as the protocol has support for a URI scheme), but for the sake of this book and since we’re focusing on API design, let’s assume that the protocol we’re using is HTTP, which will simplify explanations and examples. And as long as you keep in mind that the same is true for other protocols (like FTP), then you’ll be fine. 1 © Fernando Doglio 2018 F. Doglio, REST API Development with Node.js, https://doi.org/10.1007/978-1-4842-3715-1_1 Chapter 1 REST 101 Where Did It All Start? This whole thing started with Roy Fielding, an American computer scientist born in 1965. He is one of the main authors of the HTTP protocol1 (the protocol upon which the entire Web infrastructure is based). He is also one of the co-authors of the Apache Web server2 and he was the chair of the Apache Software Foundation3 for the first 3 years of its existence. So, as you can see, Fielding has made a lot of great contributions to the IT world, especially regarding the Internet, but I think that his doctoral thesis is the thing that received the most attention and made his name known among a lot of people who otherwise wouldn’t have heard of him. In the year 2000, Fielding presented his doctoral dissertation, Architectural Styles and the Design of Network-based Software Architecture4. In it he coined the term REST, an architectural style for distributed hypermedia systems. Put simply, REST (short for REpresentational State Transfer) is an architectural style defined to help create and organize distributed systems. The key word from that definition should be style, because an important aspect of REST (which is one of the main reasons books like this one exist) is that it is an architectural style—not a guideline, not a standard, nor anything that would imply that there are a set of hard rules to follow to end up having a RESTful architecture. And because it is a style, and there is no Request for Comments (RFC) out there to define it, it’s subject to misinterpretations from the people reading about it. Not only that, but some go as far as to leave parts out and implement a subset of its features, which in turn leads to a widespread and incomplete REST ideal, leaving out features that would otherwise be useful and help your system’s users. The main idea behind REST is that a distributed system, organized RESTfully, will improve in the following areas: • Performance: The communication style proposed by REST is meant to be efficient and simple, allowing a performance boost on systems that adopt it. 1See https://www.ietf.org/rfc/rfc2616.txt. 2See http://httpd.apache.org/. 3See http://www.apache.org/. 4See http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm. 2 Chapter 1 REST 101 • Scalability of component interaction: Any distributed system should be able to handle this aspect well enough, and the simple interaction proposed by REST greatly allows for this. • Simplicity of interface: A simple interface allows for simpler interactions between systems, which in turn can grant benefits like the ones previously mentioned. • Modifiability of components: The distributed nature of the system, and the separation of concerns proposed by REST (more on this in a bit), allows for components to be modified independently of each other at a minimum cost and risk. • Portability: REST is technology- and language-agnostic, meaning that it can be implemented and consumed by any type of technology (there are some constraints that I’ll go over in a bit, but no specific technology is enforced). • Reliability: The stateless constraint proposed by REST (more on this later) allows for the easier recovery of a system after failure. • Visibility: Again, the stateless constraint proposed has the added benefit of improving visibility, because any monitoring system doesn’t need to look further than a single request message to determine the full state of said request (this will become clear once I talk about the constraints in a bit). From this list, some direct benefits can be extrapolated. • A component-centric design allows you to make systems that are very fault tolerant. Having the failure of one component not affect the entire stability of the system is a great benefit for any system. • Interconnecting components is quite easy, minimizing the risks when adding new features or scaling up or down. • A system designed with REST in mind will be accessible to a wider audience, thanks to its portability (as described earlier).With a generic interface, the system can be used by a wider range of developers. To achieve these properties and benefits, a set of constraints were added to REST to help define a uniform connector interface. 3 Chapter 1 REST 101 REST Constraints According to Fielding, there are two ways to define a system. One approach is to start from a blank slate, an empty whiteboard, with no initial knowledge of the system being built or the use of familiar components until the needs are satisfied. A second approach is to start with the full set of needs for the system, and constraints are added to individual components until the forces that influence the system are able to interact in harmony with each other. REST follows the second approach. To define a REST architecture, a null-state is initially defined—a system that has no constraints whatsoever and where component differentiation is nothing but a myth—and constraints are added one by one. Client–Server The first constraint to be added is one of the most common ones on network-based architectures: client–server. A server is in charge of handling a set of services, and it listens for requests regarding said services. The requests, in turn, are made via a connector by a client system needing one of those services (see Figure 1-1). The main principle behind this constraint is the separation of concerns. It allows for the separation of front-end code (representation and possible UI-related processing of the information) from the server-side code, which should take care of storage and server side processing of the data. This constraint allows for the independent evolution of both components, offering a great deal of flexibility by letting client applications improve without affecting the server code and vice-versa. Figure 1-1. Client–Server architecture diagram 4 Stateless Chapter 1 REST 101 The constraint to be added on top of the previous one is the stateless constraint (see Figure 1-2). Communication between client and server must be stateless, meaning that each request done from the client must have all the information required for the server to understand it, without taking advantage of any stored data. This constraint represents several improvements for the underlying architecture: • Visibility: Monitoring the system becomes easy when all the information required is inside the request. • Scalability: By not having to store data between requests, the server can free resources faster. • Reliability: As mentioned earlier, a system that is stateless can recover from a failure much easier than one that isn’t, since the only thing to recover is the application itself. • Easier implementation: Writing code that doesn’t have to manage stored-state data across multiple servers is much easier to do, thus the full server-side system becomes simpler. Although at first glance this constraint might seem nothing but good, as what normally happens, there is a trade-off. On one hand, benefits are gained by the system, but on the other side, network traffic could potentially be harmed by adding a minor overhead on every request from sending repeated state information. Depending on the type of system being implemented, and the amount of repeated information, this might not be an acceptable trade-off. 5 Chapter 1 REST 101 Figure 1-2. Representation of the stateless constraint Cacheable The cacheable constraint is added to the current set of constraints (see Figure 1-3). It proposes that every response to a request must be explicitly or implicitly set as cacheable (when applicable). By caching the responses, there are some obvious benefits that get added to the architecture: on the server side, some interactions (a database request, for example) are completely bypassed while the content is cached. On the client side, an apparent improvement of performance is perceived. The trade-off with this constraint is the possibility of cached data being stale, due to poor caching rules. This constraint is, again, dependent on the type of system being implemented. 6 Chapter 1 REST 101 Figure 1-3. Representation of a client-stateless-cache-server architecture Note Figure 1-3 shows the cache as an external layer between the clients and the servers. This is only one possible implementation of it. The cache layer could be living inside the client (i.e., browser cache) or inside the servers themselves. Uniform Interface One of REST’s main characteristics and winning points when compared to other alternatives is the uniform interface constraint. By keeping a uniform interface between components, you simplify the job of the client when it comes to interacting with your system (see Figure 1-4). Another major winning point here is that the client’s implementation is independent of yours, so by defining a standard and uniform interface for all of your services, you effectively simplified the implementation of independent clients by giving them a clear set of rules to follow. Said rules are not part of the REST style, but there are constraints that can be used to create such rules for each individual case. 7 Chapter 1 REST 101 This benefit doesn’t come without a price, though; as with many other constraints, there is a trade-off here: having a standardized and uniform interface for all interactions with your system might harm performance when a more optimized form of communication exists. Particularly, the REST style is designed to be optimized for the Web, so the more you move away from that, the more inefficient the interface can be. Figure 1-4. Different client types can interact seamlessly with servers thanks to the uniform interface Note To achieve the uniform interface, a new set of constraints must be added to the interface: identification of resources, manipulation of resources through representation, self-descriptive messages, and hypermedia as the engine of application state (a.k.a. HATEOAS). I’ll discuss some of these constraints shortly. 8 Layered System Chapter 1 REST 101 REST was designed with the Internet in mind, which means that an architecture that follows REST is expected to work properly with the massive amount of traffic that exists in the web of webs. To achieve this, the concept of layers is introduced (see Figure 1-5). By separating components into layers, and allowing each layer to only use the one below and to communicate its output to the one above, you simplify the system’s overall complexity and keep component coupling in check. This is a great benefit in all type of systems, especially when the complexity of such a system is ever-growing (e.g., systems with massive amounts of clients, systems that are currently evolving, etc.). The main disadvantage of this constraint is that for small systems, it might add unwanted latency into the overall data flow, due to the different interactions between layers. Figure 1-5. Example of a multilayered architecture 9 Chapter 1 REST 101 Code-on-Demand Code-on-demand is the only optional constraint imposed by REST, which means that an architect using REST can choose whether or not to use this constraint and either gains its advantages or suffers its disadvantages. With this constraint, the client can download and execute code provided by the server (such as Java applets, JavaScript scripts, etc.). In the case of REST APIs (on which this book focuses), this constraint seems unnecessary, because the normal thing for an API client to do is just get information from an endpoint, and then process it however needed; but for other uses of REST, like web servers, a client (i.e., a browser) will probably benefit from this constraint (see Figure 1-6). Figure 1-6. How some clients might execute the code-on-demand, whereas others might not All of these constraints provide a set of virtual walls within which an architecture can move and still gain the benefits of the REST design style. But let’s take a step back. I initially defined REST as a design style for representational state transfer; in other words, you transfer the state of things by using some kind of representation. But what are these “things”? The main focus of a REST architecture is the resources, the owners of the state that you’re transferring. Just like in a real state (almost), it’s all about resources, resources, resources. 10 Chapter 1 REST 101 Resources, Resources, Resources The main building blocks of a REST architecture are the resources. Anything that can be named can be a resource (a web page, an image, a person, a weather service report, etc.). Resources define what the services are going to be about, the type of information that is going to be transferred, and their related actions. The resource is the main entity from which everything else is born. A resource is the abstraction of anything that can be conceptualized (from an image file, to a plain text document). The structure of a resource is shown in Table 1-1. Table 1-1. Resource Structure Description Property Description Representations It can be any way of representing data (binary, JSON, XML, etc.). A single resource can have multiple representations. Identifier A URL that retrieves only one specific resource at any given time. Metadata Content-type, last-modified time, and so forth. Control data Is-modified-since, cache-control. Representations At its core, a representation is a set of bytes, and some metadata that describes these bytes. A single resource can have more than one representation; just think of a weather service report (which could act as a possible resource). The weather report for a single day could potentially return the following information: • The date the report is referencing • The maximum temperature for the day • The minimum temperature for the day • The temperature unit to be used • A humidity percentage • A code indicating how cloudy the day will be (e.g., high, medium, low) 11 Chapter 1 REST 101 Now that the resource’s structure is defined, here are a few possible representations of the same resource: JSON { "date": "2014-10-25", "max_temp": 25.5, "min_temp": 10.0, "temp_unit": "C", "humidity_percentage": 75.0, "cloud_coverage": "low" } XML Custom pipe-separated values: 2014-10-25|25.5|10.0|C|75.0|low And there could be many more. They all successfully represent the resource correctly; it is up to the client to read and parse the information. Even when the resource has more than one representation, it is common for clients (due to simplicity of development) to only request one of them. Unless you’re doing some sort of consistency check against the API, there is no point in requesting more than one representation of the same resource, is there? There are two very popular ways to let the client request a specific representation on a resource that has more than one. The first one directly follows the principles described by REST (when using HTTP as a basis), called content negotiation, which is part of the HTTP standard. The second one is a simplified version of this, with limited benefits. For the sake of completeness, I’ll quickly go over them both. 12 Content Negotiation Chapter 1 REST 101 As mentioned, this methodology is part of the HTTP standard,5 so it’s the preferred way according to REST (at least when focused on API development on top of HTTP). It is also more flexible and provides further advantages than the other method. It consists of the client sending a specific header with the information of the different content types (or types of representations) supported, with an optional indicator of how supported/preferred that format is. Let’s look at an example from the “Content Negotiation” page on Wikipedia6: Accept: text/html; q=1.0, text/*; q=0.8, image/gif; q=0.6, image/jpeg; q=0.6, image/*; q=0.5, */*; q=0.1 This example is from a browser configured to accept various types of resources, but preferring HTML over plain text and GIF or JPEG images over other types, but ultimately accepts any other content type as a last resort. On the server side, the API is in charge of reading this header and finding the best representation for each resource, based on the client’s preferences. Using File Extensions Even though this approach is not part of the REST proposed style, it is widely used and a fairly simple alternative to the somewhat more complex other option, so I’ll cover it anyway. During the last few years, using file extensions has become an alternative preferred over using content negotiation; it is a simpler version and it doesn’t rely on a header being sent; instead, it works with the concept of file extensions. The extension portion of the file’s name indicates the content type to the operating system and any other software trying to use it; so in the following case, the extension added to the resource’s URL (unique identifier) indicates to the server the type of representation wanted. GET /api/v1/books.json GET /api/v1/books.xml Both identifiers reference the same resource—the list of books, but they request a different representation of it. 5See http://tools.ietf.org/html/rfc7231#section-5.3. 6See https://en.wikipedia.org/wiki/Content_negotiation 13 Chapter 1 REST 101 Note This approach might seem easier to implement, and even understand, by humans, but it lacks the flexibility added by content negotiation and should only be used if there is no real need for complex cases where multiple content types might be specified with their related preference. Resource Identifier The resource identifier should provide a unique way of identification at any given moment and it should provide the full path to the resource. A classic mistake is to assume it’s the resource’s ID on the storage medium used (i.e., the ID on the database). This means that you cannot consider a simple numeric ID as a resource identifier; you must provide the full path, and because we’re basing REST on HTTP, the way to access the resource it to provide its full URI (unique resource identifier). There is one more aspect to consider: the identifier of each resource must be able to reference it unequivocally at any given moment in time. This is an important distinction, because a URI like the following might reference Harry Potter and the Half Blood Prince for a certain period of time, and then Harry Potter and the Deathly Hollows 1 year later.: GET /api/v1/books/last This renders that URI as an invalid resource ID. Instead, each book needs a unique URI that is certain to not change over time; for example: GET /api/v1/books/j-k-rowling/harry-potter-and-the-deathly-hollows GET /api/v1/books/j-k-rowling/harry-potter-and-the-half-blood-prince The identifiers are unique here, because you can safely assume that the author won’t publish more books with the same title. And to provide a valid example for getting the last book, you might consider doing something like this: GET /api/v1/books?limit=1&sort=created_at The preceding URI references the lists of books, and it asks for only one, sorted by its publication date, thus rendering the last book added. 14 Actions Chapter 1 REST 101 Identifying a resource is easy: you know how to access it and you even know how to request for a specific format (if there is more than one); but that’s not all that REST proposes. Since REST is using the HTTP protocol as a standing point, the latter provides a set of verbs that can be used to reference the type of action being done over a resource. There are other actions, aside from accessing, that a client app can take in the resources provided by an API; these depend on the service provided by the API. These actions could potentially be anything, just like the type of resources handled by the system. Still, there is a set of common actions that any system that is resource-oriented should be able to provide: CRUD (create, retrieve, update, and delete) actions. These so-called actions can be directly mapped to the HTTP verbs, but REST does not enforce a standardized way to do so. However, there are some actions that are naturally derived by the verb and others that have been standardized by the API development community over the years, as shown in Table 1-2. Table 1-2. HTTP Verbs and Their Proposed Actions HTTP Verb Proposed Action GET Access a resource in a read-only mode POST Normally used to send a new resource into the server (create action) PUT Normally used to update a given resource (update action) DELETE Used to delete a resource HEAD Not part of the CRUD actions, but the verb is used to ask if a given resource exists without returning any of its representations OPTIONS Not part of the CRUD actions, but used to retrieve a list of available verbs on a given resource (i.e., What can the client do with a specific resource?) That said, a client may or may not support all of these actions; it depends on what needs to be achieved. For instance, web browsers—a clear and common example of a REST client—only have support for GET and POST verbs from within the HTML code of a page, such as links and forms (although using the XMLHttpRequest object from JavaScript would provide support for the major verbs mentioned earlier). 15 Chapter 1 REST 101 Note The list of verbs and their corresponding actions are suggestions. For instance, there are some developers who prefer to switch PUT and POST, by having PUT add new elements and POST update them. Complex Actions CRUD actions are normally required, but they’re just a very small subset of the entire spectrum of actions that a client can do with a specific resource or set of resources. For instance, take common actions like searching, filtering, working with subresources (e.g., the books of an author, the reviews of a book, etc.), sharing a blogpost, and so forth. All of these actions fail to directly match one of the verbs that I mentioned. The first solution that many developers succumb to is to specify the action taken as part of the URL; so you might end up with things like the following: GET /api/v1/blogpost/12342/like GET /api/v1/books/search GET /api/v1/authors/filtering Those URLs break the URI principle, because they’re not referencing a unique resource at any given time; instead, they’re referencing an action on a resource (or group of resources). They might seem like a good idea at first, but in the long run, and if the system keeps on growing, there will be too many URLs, which will increase the complexity of the client using the API. So to keep things simple, use the following rule of thumb: Hide the complexity of your actions behind the “?”. This rule can apply to all verbs, not just GET, and can help achieve complex actions without compromising the URL complexity of the API. For the preceding examples, the URIs could become something like this: PUT /api/v1/blogposts/12342?action=like GET /api/v1/books?q=[SEARCH-TERM] GET /api/v1/authors?filters=[COMMA SEPARATED LIST OF FILTERS] 16 Chapter 1 REST 101 Notice how the first one changed from a GET to a PUT due to the fact that the action is updating a resource by linking it. But what happens when the “?” is not enough? When the complexity of your actions is too much for the “?”, there are some alternatives that walk the line when it comes to REST-compliant solutions. Some of them might not be ideal or even desired for your API design, but when reality hits you need to do the best you can and that also includes figuring out if your use case actually fits inside REST or if you need to go another route. Use Collection of Actions An alternative way of dealing with complex actions inside your API is to treat them as resource collections. Granted, this is a bit of a stretch to what an action actually is, but like I said, we’re walking the line here guys. What this provides, is a valid interface for you to send commands to your resources following a REST-like approach. For example, say you want to reboot a VM, you can do something like the following: POST /virtual-machines//reboots { "When": "now" } This would effectively send a reboot command to your VM, telling it to reboot right now. Notice the use of the POST verb here; there are couple reasons why I used that one and not others: 1. Since we’re treating actions as resources, executing a new action is the equivalent of creating a new resource inside the action’s collection 2. POST is not an idempotent operation, which fits this use case, since we can’t be certain about the side effects your complex actions could have. You could even extend this concept a bit further and provide an actual functionality to other verbs on this endpoint, such as GET, so something like this: GET /virtual-machines//reboots 17 Chapter 1 REST 101 would provide an output detailing the list of reboots created or, put another way, a list of times the action was executed: { "reboots": [ { "received": "2001-10-23 13:30:00+0100", "params": { "when": "now" } }, { "received": "2001-12-03 03:30:00+0100", "params": { "when": "5 min" } }, { "received": "2002-10-23 13:30:00+0100", "params": { "when": "now" } } ] } Furthermore, each action could provide an endpoint to return information about it, such as documentation to help developers using them. For instance, a request such as: GET /virtual-machines//reboots/1 could return something like: { "last-executed": "2018-02-23 15:02:02+0300", "params": { "when": { "Required": true, "Docs": “Describes the system reboot needs to take place, relative to the moment the action is processed” } } } Another classic problematic endpoint that REST APIs tend to have is the so-called “search endpoint,” one that can’t always fit inside our resource-based schema. And for this, we have a couple of options. 18 Single-Entity Searches Chapter 1 REST 101 If the search functionality is only going to return, and overall deal with, one kind of entity, or if you’re going to provide scoped searches (meaning you will always be searching on specific collections), then you can probably hide the search feature behind the listing endpoint and use GET requests with search parameters GET /books/?q=Harry pott&p=1&size=10 The previous example simply hijacks the listing endpoint and adds the ability to filter the results but accepting a search string. These parameters could even be optional, rendering this endpoint quite powerful and versatile. Note The endpoint you define for your search functionality does not ensure the type of results you’ll get, that will depend on the underlying technology used. If you’re looking for simple searches, a quick SQL query can give you the results you’re looking for. But if you need something more complex, I would suggest looking into solutions such as Elastic. Multi-Entity Searches Now, if instead of just looking for books, you actually want your user to look for books and also for author’s bios, and client names. All at the same time and have those results all mixed in as one single list. Then there is no real easy way to relate this type of search to a single resource, is there? In this case, the only option that we have is to bite the bullet and break the mantra; in other words, create an endpoint that references the actual action instead of a single resource, i.e: GET /search?q=Harry&p=1&size=10 If you can, and as long as it makes sense, always try to use endpoints such as GET/action to provide information about the action and POST /action to perform it. In the end, if you end up having to “bite the bullet” a lot while designing the endpoints of your system, then maybe REST is not the best paradigm for you. 19 Chapter 1 REST 101 Hypermedia in the Response and Main Entry Point To make REST’s interface uniform, several constraints must be applied. One of them is Hypermedia as the Engine of Application State, also known as HATEOAS. I’ll go over what that concept means, how it is meant to be applied by a RESTful system, and finally, how you end up with a great new feature that allows any RESTful system client to start the interaction knowing only a single endpoint of the entire system (the root endpoint). Again, the structure of a resource contains a section called metadata; inside that section, the representation of every resource should contain a set of hypermedia links that let the client know what to do with each resource. By providing this information in the response itself, the next steps any client can take are there, thus providing an even greater level of decoupling between client and server. Changes to the resource identifiers, or added and removed functionalities, can be provided through this method without affecting the client at all, or at worst, with minimal impact. Think of a web browser: all it needs to help a user navigate through a favorite site is the home page URL; after that, the following actions are presented inside the representation (HTML code) as links. Those are the only logical next steps that the user can take, and from there, new links will be presented, and so on. In the case of a RESTful service, the same thing can be said: by calling upon the main endpoint (also known as bookmark or root endpoint), the client will discover all possible first steps (normally things like resource lists and other relevant endpoints). Let’s look at an example in Listing 1-1. Root endpoint: GET /api/v1/ Listing 1-1. Example of a JSON Response from the Root Endpoint { "metadata": { "links": { "books": { "uri": "/books", "content-type": "application/json" }, "authors": { "uri": "/authors", 20 Chapter 1 REST 101 "content-type": "application/json" } } } } Now let’s look at the books list endpoint’s results in Listing 1-2: GET /api/v1/books Listing 1-2. Example of Another JSON Response with Hyperlinks to Other Resources { "resources": [ { "title": "Harry Potter and the Half Blood prince", "description": "......", "author": { "name": "J.K.Rowling", "metadata": { "links": { "data": { "uri": "/authors/j-k-rowling", "content-type": "application/json" }, "books": { "uri": "/authors/j-k-rowling/books", "content-type": "application/json" } } } }, "copies": 10 }, { "title": "Dune", "description": "......", "author": { 21 Chapter 1 REST 101 "name": "Frank Herbert", "metadata": { "links": { "data": { "uri": "/authors/frank-herbert", "content-type": "application/json" }, "books": { "uri": "/authors/frank-herbert/books", "content-type": "application/json" } } } }, "copies": 5 } ], "total": 100, "metadata": { "links": { "next": { "uri": "/books?page=1", "content-type": "application/json" } } } } There are three sections highlighted in preceding example (Listing 1-2); those are the links returned on the response. With that information, the client application knows the following logical steps: 1. How to get the information from the books authors 2. How to get the list of books by the authors 3. How to get the next page of results 22 Chapter 1 REST 101 Note that the full list of authors is not accessible through this endpoint; this is because for this particular use case, it’s not needed, so the API just doesn’t return it. It was present on the root endpoint, though, so if the client needs it when displaying the information to the end-user, it should still have it available. Each link from the preceding example contains an attribute specifying the content type of the representation of that resource. If the resources have more than one possible representation, the different formats could be added as different links inside each resource’s metadata element, letting the client choose the most adequate to the current use case, or the type could change based on the client’s preferences (content negotiation). Note that the earlier JSON structure (more specifically, the metadata elements’ structure) is not important. The relevant part of the example is the information presented in the response. Each server has the freedom to design the structure as needed. Not having a standard structure might harm the developer experience while interacting with your system, so it might be a good idea to adopt one. This is certainly not enforced by REST, but it would be a major point in favor of your system. A good standard to adopt in this case would be Hypertext Application Language, or HAL,7 which tries to create a standard for both XML and JSON when representing resources with those languages. A Few Notes on HAL HAL tries to define a representation as having two major elements: resources and links. According to HAL, a resource has links, embedded resources (other resources associated to their parent), and a state (the actual properties that describe the resource). On the other hand, links have a target (the URI), a relation, and some other optional properties to deal with deprecation, content negotiation, and so forth. 7See http://stateless.co/hal_specification.html. 23 Chapter 1 REST 101 Listing 1-3 shows the preceding example represented using the HAL format. Listing 1-3. JSON Response Following the HAL Standard { "_embedded": [ { "title": "Harry Potter and the Half Blood prince", "description": "......", "copies": 10, "_embedded": { "author": { "name": "J.K.Rowling", "_links": { "self": { "href": "/authors/j-k-rowling", "type": "application/json+hal" }, "books": { "href": "/authors/j-k-rowling/books", "type": "application/json+hal" } } } } }, { "title": "Dune", "description": "......", "copies": 5, "_embedded": { "author": { "name": "Frank Herbert", 24 Chapter 1 REST 101 "_links": { "self": { "href": "/authors/frank-herbert", "type": "application/json+hal" }, "books": { "href": "/authors/frank-herbert/books", "type": "application/json+hal" } } } } } ], "total": 100, "_links": { "self": { "href": "/books", "type": "application/json+hal" }, "next": { "href": "/books?page=1", "type": "application/json+hal" } } } The main change in Listing 1-3 is that the actual books have been moved inside an element called "_embedded", as the standard dictates, since they’re actual embedded documents inside the represented resource, which is the list of books (the only property that belongs to the resource is "total", representing the total number of results). The same can be said for the authors, now inside the "_embedded" element of each book. 25 Chapter 1 REST 101 Status Codes Another interesting standard that REST can benefit from when based on HTTP is the usage of HTTP status codes.8 A status code is a number that summarizes the response associated to it. There are some common ones, like 404 for “Page not found,” or 200 for “OK,” or the always helpful 500 for “Internal server error” (that was irony, in case it wasn’t clear enough). A status code is helpful for clients to begin interpreting the response, but in most cases, it shouldn’t be a substitute for it. As the API owner, you can’t really transmit in the response what exactly caused a crash on your side by just replying with the number 500. There are some cases, though, when a number is enough, like 404; although a good response will always return information that should help the client solve the problem (with a 404, a link to the home page or the root URL are good places to start). These codes are grouped in five sets, based on their meaning: • 1xx: Informational and only defined under HTTP 1.1. • 2xx: The request went OK, here’s your content. • 3xx: The resource was moved somehow to somewhere. • 4xx: The source of the request did something wrong. • 5xx: The server crashed due to some error in its code. With that in mind, Table 1-3 lists some classic status codes that an API could potentially use. 8See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html. 26 Chapter 1 REST 101 Table 1-3. HTTP Status Codes and Their Related Interpretation Status Code Meaning 200 OK. The request went fine and the content requested was returned. This is normally used on GET requests. 201 Created. The resource was created and the server has acknowledged it. It could be useful on responses to POST or PUT requests. Additionally, the new resource could be returned as part of the response body. 204 No content. The action was successful but there is no content returned. Useful for actions that do not require a response body, such as a DELETE action. 301 Moved permanently. This resource was moved to another location and the location is returned. This header is especially useful when URLs change over time (maybe due to a change in version, a migration, or some other disruptive change), keeping the old ones and returning a redirection to the new location allows old clients to update their references in their own time. 400 Bad request. The request issued has problems (for example, might be lacking some required parameters). A good addition to a 400 response might be an error message that a developer can use to fix the request. 401 Unauthorized. Especially useful for authentication when the requested resource is not accessible to the user owning the request. 403 Forbidden. The resource is not accessible, but unlike 401, authentication will not affect the response. 404 Not found. The URL provided does not identify any resource. A good addition to this response could be a set of valid URLs that the client can use to get back on track (root URL, previous URL used, etc.). 405 Method not allowed. The HTTP verb used on a resource is not allowed—for instance, doing a PUT on a resource that is read-only. 500 Internal server error. A generic error code when an unexpected condition is met and the server crashes. Normally, this response is accompanied by an error message explaining what went wrong. 27 Chapter 1 REST 101 Note To see the full list of HTTP status codes and their meanings, please refer to the RFC of HTTP 1.1.9 REST vs. the Past Before REST was all cool and hip, and every business out there wanted to provide their clients with a RESTful API in their service, there were other options for developers who wanted to interconnect systems. These are still being used on old services or by services that required their specific features, but less and less so every year. Back in the 1990s, the software industry started to think about system interoperability and how two (or more) computers could achieve it. Some solutions were born, such as COM,10 created by Microsoft, and CORBA,11 created by the Object Management Group. These were the first two implementations back then, but they had a major issue: they were not compatible with each other. Other solutions arose, like RMI (Remote Method Invocation), but it was meant specifically for Java, which meant it was technology-dependent, and hadn’t really caught up with the development community. By 1997, Microsoft decided to research solutions that would use XML as the main transport language and would allow systems to interconnect using RPC (Remote Procedure Call) over HTTP, thus achieving a somewhat technology-independent solution that would considerably simplify system interconnectivity. That research gave birth to XML-RPC around 1998. Listing 1-4 is a classic XML-RPC request taken from Wikipedia:12 9See http://tools.ietf.org/html/rfc7231#section-6. 10See https://en.wikipedia.org/wiki/Component_Object_Model 11See http://www.corba.org/. 12See http://en.wikipedia.org/wiki/XML-RPC. 28 Chapter 1 REST 101 Listing 1-4. Example of an XML-RPC Request examples.getStateName 40 Listing 1-5 shows a possible response. Listing 1-5. Example of an XML-RPC Response South Dakota From the examples shown in Listing 1-4 and Listing 1-5, it is quite clear that the messages (both requests and responses) were overly verbose, something that was directly related to the use of XML. There are implementations of XML-RPC that exist today for several operating systems and programming languages, like Apache XML RPC13 (written in Java), XMLRPC-EPI14 (written in C), and XML-RPC-C15 for C and C++ (see Figure 1-7). 13See http://ws.apache.org/xmlrpc/. 14See http://xmlrpc-epi.sourceforge.net. 15See http://xmlrpc-c.sourceforge.net/. 29 Chapter 1 REST 101 Figure 1-7. Diagram showing the basic architecture of an XML-RPC interaction After XML-RPC became more popular, it mutated into SOAP,16 a more standardized and formalized version of the same principle. SOAP still uses XML as the transport language, but the message format is now richer (and therefore complex). Listing 1-6 is an example from W3C’s specification page on SOAP: Listing 1-6. Example of a SOAP Request uuid:093a2da1-q345-739r-ba5d-pqff98fe8j7d 2001-11-29T13:20:00.000-05:00 16See http://www.w3.org/TR/soap/. 30 Chapter 1 REST 101 Åke Jógvan Øyvind New York Los Angeles 2001-12-14 late afternoon aisle Los Angeles New York 2001-12-20 mid-morning none 31 Chapter 1 REST 101 Figure 1-8 shows the basic structure of the example from Listing 1-6. Figure 1-8. Image from the W3C SOAP spec page SOAP services are actually dependent on another technology called Web Service Description Language (WSDL). An XML-based language, it describes the services provided to clients that want to consume them. Listing 1-7 is an annotated WSDL example taken from the W3C web site.17 Listing 1-7. WSDL Example 33 Chapter 1 REST 101 34 Chapter 1 REST 101 snowboarding-info.com Endorsement Service 35 Chapter 1 REST 101 The main drawback of these types of services was the amount of information used, both to describe them and to use them. Even though XML provided the much-required technology agnostic means of encoding data to be transmitted between two systems, it also blotted the message sent quite noticeably. Both of these technologies (XML-RPC and SOAP + WSDL) provided the solution to system interconnectivity at a time when it was required. They provided a way to transmit messages using a “universal” language between all systems, but they also had several major issues compared to today’s leading standard (see Table 1-4). This can be clearly seen, for example, in the way developers feel about using XML instead of JSON. Table 1-4. Comparison of XML-RPC/SOAP and REST Services XML-RCP/SOAP REST Specific SOAP clients had to be created for each programming language. Even though XML was universal, a new client would have to be coded to parse the WSDL to understand how the service worked. The client needs to know everything about the service before initiating the interaction (thus the WSDL mentioned earlier). Because the service was used from within the client source code and called a specific function or method from within the server’s code, the coupling between those two systems was too big. A rewrite of the server code would probably lead to a rewrite on the client’s code. REST is completely technology-agnostic and doesn’t require special clients, only a programming language capable of connectivity through the chosen protocol (e.g., HTTP, FTP, etc.). The client only needs to know the main root endpoint, and with the hypermedia provided on the response, self-discovery is possible. The interface is implementation independent; the complete server-side code can be rewritten and the API’s interface will not have to be changed. 36 Chapter 1 REST 101 Note Comparing XML-RPC/SOAP with REST might not be entirely fair (or possible) due to the fact that the first two are protocols, whereas the latter is an architectural style; but some points can still be compared if you keep that distinction in mind. Summary This chapter was a small overview of what REST is meant to be and the kind of benefits a system will gain by following the REST style. The chapter also covered a few extra principles, like HTTP verbs and status codes, which despite not being part of the REST style, are indeed part of the HTTP standard, the protocol we’re basing this book on. Finally, I discussed the main technologies used prior to REST, and you saw how they compared to the current leading industry standard. In the next chapter, I’ll go over some good API design practices, and you’ll see how you can achieve them using REST. 37 CHAPTER 2 API Design Best Practices The practice of API design is a tricky one. Even when there are so many options out there—tools to use, standards to apply, styles to follow—there is one basic question that needs to be answered and needs to be clear in the developer’s mind before any kind design and development can begin… What Defines a Good API? As we all know, the concepts of “good” and “bad” are very subjective (one could probably read a couple of books discussing this on its own), and therefore, opinions vary from one person to another. That being said, years of experience of dealing with different kinds of APIs have left the developer community (and this author) with a pretty good sense of the need-to-have features of any good API. (Disclaimer: Things like clean code, good development practices, and other internal considerations will not be mentioned here but will be assumed, since they should be part of every software undertaking.) So let’s go over this list. • Developer-friendly: The developers working with your API should not suffer when dealing with your system. • Extensibility: Your system should be able to handle the addition of new features without breaking your clients. • Up-to-date documentation: Good documentation is key to your API being picked up by new developers. • Proper error handling: Because things will go wrong and you need to be prepared. • Provides multiple SDK/libraries: The more work you simplify for developers, the more they’ll like your system. 39 © Fernando Doglio 2018 F. Doglio, REST API Development with Node.js, https://doi.org/10.1007/978-1-4842-3715-1_2 Chapter 2 API Design Best Practices • Security: A key aspect of any global system. • Scalability: The ability to scale up and down is something any good API should have to properly provide its services. I’ll go over these points one by one and show how they affect the API and how following the REST style help. Developer-Friendly By definition, an API is an application programming interface, with the key word being interface. When thinking about designing an API that will be used by developers other than yourself, there is a key aspect that needs to be taken into consideration: the Developer eXperience (or DX). Even when the API will be used by another system, the integration into that system is first done by one or more developers—human beings that bring the human factor into that integration. This means you’ll want the API to be as easy to use as possible, which makes for a great DX, and which should translate into more developers and client applications using the API. There is a trade-off, though, since simplifying things for humans could lead into an oversimplification of the interface, which in turn could lead to design issues when dealing with complex functionalities. It is important to consider the DX as one of the major aspects of an API (let’s be honest, without developers using it, there is no point to an API), but there are other aspects that have to be considered and given weight in the design decisions. Make it simple, but not dummy simple. The next sections provide some pointers for a good DX. Communication’s Protocol This is one of the most basic aspects of the interface. When choosing a communication protocol, it’s always a good idea to go with one that is familiar to the developers using the API. There are several standards that already have libraries and modules available in many programming languages (e.g., HTTP, FTP, SSH, etc.). A custom-made protocol isn’t always a good idea because you’ll lose that instant portability in so many existing technologies. That said, if you’re ready to create support 40 Chapter 2 API Design Best Practices libraries for the most used languages, and your custom protocol is more efficient for your use case, it could be the right choice. In the end, it’s up to the API designer to evaluate the best solution based on the context in which he’s working. In this book, you’re working under the assumption that the protocol chosen for REST is HTTP.1 It’s a very well-known protocol; any modern programming language supports it and it’s the basis for the entire Internet. You can rest assured that most developers have a basic understanding of how to use it. And if not, there is plenty of information out there to get to know it better. In summary, there is no silver bullet protocol out there perfect for every scenario. Think about your API needs, make sure that whatever you choose is compatible with REST, and you’ll be fine. Easy-to-Remember Access Points The points of contact between all client apps and the API are called endpoints. The API needs to provide them to allow clients to access its functionalities. This can be done through whatever communications protocol is chosen. These access points should have mnemotechnic names to help the developer understand their purpose just by reading them. Of course, the name by itself should never be a replacement for a detailed documentation, but it is normally considered a good idea to reference the resource being used and to have some kind of indicator of the action being taken when calling that access point. The following is a good example of a badly named access point (meant to list the books in a bookstore): GET /books/action1 This example uses the HTTP protocol to specify the access point, and even though the entity used (books) is being referenced, the action name is not clear; action1 could mean anything, or even worse, the meaning could change in the future, but the name would still be suitable, so any existing client would undoubtedly break. A better example—one that follows REST and the standards discussed in Chapter 1—would be this: GET /books 1See http://www.w3.org/Protocols/rfc2616/rfc2616.html. 41 Chapter 2 API Design Best Practices This should present the developer with more than enough information to understand that a GET request into the root of a resource (/books) will always yield a list of items of this type; then the developer can replicate this pattern into other resources, as long as the interface is kept uniform across all other endpoints. Uniform Interface Easy-to-remember access points are important, but so is being consistent when defining them. Again, you have to go back to the human factor when consuming an API: you’re a human too. So making the lives of the developers using your APIs easier is a must if you want anyone to use it, you can’t forget about the DX. That means you need to be consistent when defining endpoints’ names, request formats, and response formats. There can be more than one version for the latter two (more specifically, the response format is directly tied to the various representations a resource can have), but as long as the default is always the same, there will be no problems. A good example of an inconsistent interface, even though not on an API, can be seen in the programming language PHP. It has underscore notation on most functions’ names, but the underscore is not used on some, so the developer is always forced to go back to the documentation to check how to write these functions (or worse, rely on his/her memory). For example, str_replace is a function that uses an underscore to separate both words (str and replace), whereas htmlentities has no separation of words at all. Another example of bad design practice in an API is to name the endpoints based on the actions taken instead of the resources handled; for example: /getAllBooks /submitNewBook /updateAuthor /getBooksAuthors /getNumberOfBooksOnStock These examples clearly show the pattern that this API is following. And at first glance, they might not seem that bad, but consider how poor the interface is going to become as new features and resources are added to the system (not to mention if the actions are modified). Each new addition to the system causes extra endpoints to the API’s interface. The developers of client apps will have no clue as to how these new endpoints are 42 Chapter 2 API Design Best Practices named. For instance, if the API is extended to support the cover images of books, with the current naming scheme, these are all possible new endpoints: /addNewImageToBook /getBooksImages /addCoverImage /listBooksCovers And the list can go on. So for any real-world application, you can safely assume that following this type of pattern will yield a really big list of endpoints, increasing the complexity of both server-side code and client-side code. It will also hurt the system’s ability to capture new developers, due to the inherited complexity that it will have over the years. To solve this problem and generate an easy-to-use and uniform interface across the entire API, you can apply the REST style to the endpoints. If you remember the constraints proposed by REST from Chapter 1, you end up with a resource-centric interface. And thanks to HTTP, you also have verbs to indicate actions. Table 2-1 shows how the previous interface changes using REST. Table 2-1. List of Endpoints and How They Change When the REST Style Is Applied Old Style REST Style /getAllBooks GET /books /submitNewBook POST /books /updateAuthor PUT /authors/:id /getBooksAuthors GET /books/:id/authors /getNumberOfBooksOnStock GET /books (This number can easily be returned as part of this endpoint.) /addNewImageToBook PUT /books/:id /getBooksImages GET /books/:id/images /addCoverImage POST /books/:id/cover_image /listBooksCovers GET /books (This information can be returned in this endpoint using subresources.) 43 Chapter 2 API Design Best Practices You went from having to remember nine different endpoints to just two, with the added bonus of having all HTTP verbs being the same in all cases once you defined the standard; now there is no need to remember specific roles in each case (they’ll always mean the same thing). Transport Language Another aspect of the interface to consider is the transport language used. For many years, the de facto standard was XML; it provided a technology-agnostic way of expressing data that could easily be sent between clients and servers. Nowadays, there is a new standard gaining popularity over XML—JSON. Why JSON? JSON has been gaining traction over the past few years (see Figure 2-1) as the standard Data Transfer Format. This is mainly due to the advantages that it provides. The following lists just a few: • It’s lightweight. There are very little data in a JSON file that are not directly related to the information being transferred. This is a major winning point over more verbose formats like XML.2 • It’s human-readable. The format itself is so simple that it can easily be read and written by a human. This is particularly important considering that a focus point of the interface of any good API is the human factor (otherwise known as the DX). • It supports different data types. Because not everything being transferred is a string, this feature allows the developer to provide extra meaning to the information transferred. 2XML is not strictly a Data Transfer Format, but it’s being used as one. 44 Chapter 2 API Design Best Practices Figure 2-1. Trend of Google searches for “JSON” vs. “XML” over the last few years The list could go on, but these are the three main aspects that are helping JSON win so many followers in the developer community. Even though JSON is a great format and is gaining traction, it’s not the silver bullet that will always solve all of your problems, so it’s also important to provide clients with options. And here is where REST comes to help. Since the protocol you’re basing REST on is HTTP, developers can use a mechanism called content negotiation to allow clients to specify which of the supported formats they want to receive (as discussed in Chapter 1). This allows for more flexibility on the API and still keeps the interface uniform. Going back to the list of endpoints, the last one talks about using a subresource as the solution. That can be interpreted in several ways, because not only is the language used to transfer the data important, but so is the structure that you give the data being transferred. My final advice for a uniform interface is to standardize the format used, or even better, follow an existing one, like Hypertext Application Language (HAL). This was covered in Chapter 1, so refer back to it for more information. 45 Chapter 2 API Design Best Practices Extensibility A good API is never fully finished. This might be a bold claim, but it’s one that comes from the experience of the community. Let’s look at some of the big ones.3 • Google APIs: 5 billion calls a day;4 launched in 2005 • Facebook APIs: 5 billion calls a day;5 launched in 2007 • Twitter APIs: 13 billion calls a day;6 launched in 2006 These examples show that even when a great team is behind the API, the APIs will keep growing and changing because the client apps developers find new ways to use it, the business model of the API owner changes over time, or simply because features are added and removed. When any of this happens, the API may need to be extended or changed, and new access points added or old ones changed. If the original design is right, then going from v1 to v2 should be no problem, but if it’s not, then that migration could spell disaster for everyone. How Is Extensibility Managed? When extending the API, you’re basically releasing a new version of your software, so the first thing you need to do is let your users (the developers) know what will happen once the new version is out. Will their apps still work? Are the changes backward-compatible? Will you maintain several versions of your API online or just the latest one? A good API should take the following points into consideration: • How easily can new endpoints be added? • Is the new version backward-compatible? • Can clients continue to work with older versions of the API while their code is being updated? • What will happen to existing clients targeting the new API? 3Source: http://www.slideshare.net/3scale/apis-for-biz-dev-20-which-business-model-15473323. 4Source: http://www.slideshare.net/3scale/apis-for-biz-dev-20-which-business-model-15473323, April 2010. 5Source: http://www.slideshare.net/3scale/apis-for-biz-dev-20-which-business-model-15473323, October 2009. 6Source: http://www.slideshare.net/3scale/apis-for-biz-dev-20-which-business-model-15473323, May 2011. 46 Chapter 2 API Design Best Practices • How easy will it be for clients to target the new version of the API? Once all these points are settled, then you can safely grow and extend the API. Normally, going from version A to version B of an API by instantly deprecating version A and taking it offline in favor of version B is considered a bad move, unless, of course, you have very few client applications using that version. A better approach for this type of situation is to allow developers to choose which version of the API they want to use, keeping the old version long enough to let everyone migrate into the newer one. And to do this, an API would include its version number in the resource identifier (i.e., the URL of each resource). This approach makes the version number a mandatory part of the URL to clearly show the version in use. Another approach, which may not be as clear, is to provide a versionless URL that points to the latest version of the API and an optional URL parameter to overwrite the version. Both approaches have pros and cons that have to be weighted by the developer creating the API. Tables 2-2 and 2-3 show the pros and cons of both options. Table 2-2. Pros and Cons of Having the Version of the API As Part of the URL Pros Cons The version number is clearly visible, helping prevent confusion about the version being used. Easy to migrate from one version to another, from a client perspective (all URLs change the same portion—the version number ) Allows cleaner architecture when more than one version of the API needs to be kept working Clear and simple migration from one version to the next from the API perspective, since both versions could be kept working in parallel for a period of time, allowing slower clients to migrate without breaking The right versioning scheme can make fixes and backward-compatible new features instantly available without the need to update on the client’s part. URLs are more verbose. A wrong implementation on the API code could cause a huge amount of work when migrating from one version to the other (i.e., if the version is hardcoded on the endpoint’s URL template, individually for every endpoint). 47 Chapter 2 API Design Best Practices Table 2-3. Pros and Cons of Having the API Version Hidden from the User Pros Cons Simpler URLs A Hidden version number might lead to confusion about the version being used. Instant migration to latest working code of the API Simple migration from one version to the next from the client’s perspective (only change the value of the attribute) Easy test of client code against the latest version (just don’t send version-specific parameters) Non-backward-compatible changes will break the clients that are not referencing a specific version of the API. Complex architecture required to make version selection available Keeping this in mind, there are several versioning schemes to use when it comes to setting the version of a software product: • Ubuntu’s7 version numbers represent the year and month of the release; so version 14.04 means it was released in April 2014. • In the Chromium project, version numbers have four parts:8 MAJOR. MINOR.BUILD.PATCH. The following is from the Chromium project’s page on versioning: MAJOR and MINOR may get updated with any significant Google Chrome release (Beta or Stable update). MAJOR must get updated for any backward-incompatible user data change (since this data survives updates). BUILD must get updated whenever a release candidate is built from the current trunk (at least weekly for Dev channel release candidates). The BUILD number is an ever-increasing number representing a point in time of the Chromium trunk. PATCH must get updated whenever a release candidate is built from the BUILD branch. 7See https://help.ubuntu.com/community/CommonQuestions#Ubuntu_Releases_and_Version_Numbers. 8See http://www.chromium.org/developers/version-numbers. 48 Chapter 2 API Design Best Practices • Another intermediate approach, known as Semantic Versioning or SemVer,9 is well accepted by the development community. It provides the right amount of information. It has three numbers for each version: MAJOR.MINOR.PATCH. • MAJOR represents changes that are not backward-compatible. • MINOR represents new features that leave the API backward compatible. • PATCH represents small changes like bug fixes and code optimization. With that scheme, the first number is the only one that is really relevant to clients, since that’ll be the one indicating compatibility with their current version. By having the latest version of MINOR and PATCH deployed on the API at all times, you’re providing clients with the latest compatible features and bug fixes, without making clients update their code. So with that simple versioning scheme, the endpoints look like this: GET /1/books?limit=10&size=10 POST /v2/photos GET /books?v=1 When choosing a versioning scheme, please take the following into consideration: • Using the wrong versioning scheme might cause confusion or problems when implementing a client app by consuming the wrong version of the API. For instance, using Ubuntu’s versioning scheme for your API might not be the best way to communicate what is going on in each new version. • The wrong versioning scheme might force clients to update a lot, like when a minor fix is deployed or a new backward-compatible feature is added. Those changes shouldn’t require a client update. So don’t force the client to specify those parts of the version unless your scheme requires it. 9See semver.org. 49 Chapter 2 API Design Best Practices Up-to-Date Documentation No matter how mnemotechnic your endpoints are, you still need to have documentation explaining everything that your API does. Whether optional parameters or the mechanics of an access point, the documentation is fundamental to having a good DX, which translates into more users. A good API requires more than just a few lines explaining how to use an access point (there is nothing worse than discovering that you need an access point but it has no documentation at all) but needs a full list of parameters and explanatory examples. Some providers give developers a simple web interface to try their API without having to write any code. This is particularly useful for newcomers. There are some online services that allow API developers to upload their documentation, as well as those that provide the web UI to test the API; for example, Mashape provides this service for free (see Figure 2-2). Figure 2-2. The service provided by Mashape 50 Chapter 2 API Design Best Practices Another good example of detailed documentation is at Facebook’s developer site.10 It provides implementation and usage examples for all the platforms that Facebook supports (see Figure 2-3). Figure 2-3. Facebook’s API documentation site An example of a poorly written documentation is seen in Figure 2-4. It is 4chan’s API documentation.11 Yes, the API appears to not be complicated enough to merit writing a whole book about it, but then again, there are no examples provided, only a generic explanation of how to find the endpoints and what parameters to use. Newcomers might find it hard to understand how to implement a simple client that uses this API. 10See https://developers.facebook.com/docs/graph-api/using-graph-api/v2.1. 11See https://github.com/4chan/4chan-API. 51 Chapter 2 API Design Best Practices Note It’s unfair to compare 4chan’s documentation to that of Facebook’s, since the size of the teams and companies are completely different. But you should note the lack of quality in 4chan’s documentation. Figure 2-4. Introduction to 4chan’s API documentation Although it might not seem like the most productive idea while developing an API, the team needs to consider working on extensive documentation. It is one of the main things that will assure the success or failure of the API for two main reasons: • It should help newcomers and advance developers to consume your API without any problems. • It should serve as a blueprint for the development team, if it is kept up-to date. Jumping into a project mid-development is easier if there is a well written and well-explained blueprint of how the API is meant to work. 52 Chapter 2 API Design Best Practices Note This also applies to updating the documentation when changes are made to the API. You need to keep it updated; otherwise, the effect is the same as not having documentation at all. Proper Error Handling Error handling on an API is incredibly important, because if it is done right, it can help the client app understand how to handle errors; and on the human side (the DX), it can help developers understand what it is they’re doing wrong and how to fix it. There are two very distinct moments during the life cycle of an API client that you need to consider error handling: • Phase 1: The development of the client • Phase 2: The client is implemented and being used by end users. Phase 1: Development of the Client During the first phase, developers implement the required code to consume the API. It is very likely that a developer will have errors on the requests (things like missing parameters, wrong endpoint names, etc.) during this stage. Those errors need to be handled properly, which means returning enough information to let developers know what they did wrong and how they can fix it. 53 Chapter 2 API Design Best Practices A common problem with some systems is that their creators ignore this stage, and when there is a problem with the request, the API crashes, and the returned information is just an error message with the stack trace and the status code 500, as seen in Figure 2-5. Figure 2-5. A classic example of a crash on the API returning the stack trace The response in Figure 2-5 shows what happens when you forget to add error handling in the client development stage. The stack trace returned might give the developer some sort of clue (at best) as to what exactly went wrong, but it also shows a lot of unnecessary information, so it ends up being confusing. This certainly hurts development time, and no doubt would be a major point against the DX of the API. 54 Chapter 2 API Design Best Practices On the other hand, let’s take a look at a proper error response for the same error in Figure 2-6. Figure 2-6. A proper error response would look like this Figure 2-6 clearly shows that there has been an error, what the error is, and an error code. The response only has three attributes, but they’re all helpful: • The error indicator gives the developer a clear way to check whether or not the response is an error message (you could also check against the status code of the response). • The error message is clearly intended for the developer and not only states what’s missing, but also explains how to fix it. • A custom error code, if explained in the documentation, could help a developer automate actions when this type of response happens again. Phase 2: The Client Is Implemented and Being Used by End Users During this stage in the life cycle of the client, you’re not expecting any more developer errors, such as using the wrong endpoint, missing parameters, and the like, but there could still be problems caused by the data generated by the user. Client applications that request some kind of input from the user are always subject to errors on the user’s part, and even though there are always ways to validate that input before it reaches the API layer, it’s not safe to assume all clients will do that. So the safest bet for any API designer and developer is to assume there is no validation done by the 55 Chapter 2 API Design Best Practices client, and anything that could go wrong with the data will go wrong. This is also a safe assumption to make from a security point of view, so it’s providing a minor security improvement as a side effect. With that mindset, the API implemented should be rock-solid and able to handle any type of errors in the input data. The response should mimic that from phase 1: there should be an error indicator, an error message stating what’s wrong (and, if possible, how to fix it), and a custom error code. The custom error code is especially useful in this stage, since it’ll provide the client with the ability to customize the error shown to the end user (even showing a different but still relevant error message). Multiple SDK / Libraries If you expect your API to be massively used across different technologies and platforms, it might be a good idea to develop and provide support for libraries and SDKs that can be used with your system. By doing so, you provide developers with the means to consume your services, so all they have to do is use these services to create their client apps. Essentially, you’re shaving off potential weeks or months (depending on the size of your system) of development time. Another benefit is that most developers will inherently trust your libraries over others that do the same, because you’re the owner of the service those libraries are consuming. Finally, consider open sourcing the code of your libraries. These days, the open source community is thriving. Developers will undoubtedly help maintain and improve your libraries if they’re of use to them. Let’s look again at some of the biggest APIs out there: • Facebook API provides SDKs for iOS, Android, JavaScript, PHP, and Unity.12 • Google Maps API provides SDKs for several technologies, including iOS, the Web, and Android.13 12See https://developers.facebook.com (see the bottom of the page for the list of SDKs). 13See https://developers.google.com/maps/. 56 Chapter 2 API Design Best Practices • Twitter API provides SDKs for several of their APIs, including Java, ASP, C++, Clojure, .NET, Go, JavaScript, and a lot of other languages.14 • Amazon provides SDKs for their AWS service, including PHP, Ruby, .NET, and iOS. They even have those SDKs on GitHub for anyone to see.15 Security Securing your API is a very important step in the development process, and it should not be ignored, unless what you’re building is small enough and has no sensitive data to merit the effort. There are two big security issues to deal with when designing an API: • Authentication: Who’s going to access the API? • Authorization: What will they be able to access once logged in? Authentication deals with letting valid users access the features provided by the API. Authorization deals with handling what those authenticated users can actually do inside the system. Before going into details about each specific issue, there are some common aspects that need to be remembered when dealing with security on RESTful systems (at least, those based on HTTP): • RESTful systems are meant to be stateless: Remember that REST defines the server as stateless, which means that storing the user data in session after the initial login is not a good idea (if you want to stay within the guidelines provided by REST, that is). • Remember to use HTTPS: On RESTful systems based on HTTP, HTTPS should be used to assure encryption of the channel, making it harder to capture and read data traffic (man-in-the-middle attack). 14See https://dev.twitter.com/overview/api/twitter-libraries. 15See https://github.com/aws. 57 Chapter 2 API Design Best Practices Accessing the System There are some widely used authentication schemes out there meant to provide different levels of security when signing users into a system. Some of the most commonly known are Basic Auth with TSL, Digest Auth, OAuth 1.0a, and OAuth 2.0. I’ll go over these and talk about each of their pros and cons. I’ll also cover an alternative method that should prove to be the most RESTful, in the sense that it’s 100% stateless. Almost Stateless Methods OAuth 1.0a, OAuth 2.0, Digest Auth, and Basic Auth + TSL are the go-to methods of authentication these days. They work wonderfully, they have been implemented in all of the modern programming languages, and they have proven to be the right choice for the job (when used for the right use-case). That being said, as you’re about to see, none of them are 100% stateless. They all depend on having the user have information stored on some kind of cache layer on the server side. This little detail, especially for the purists out there, means a no-go when designing a RESTful system, because it goes against one of the most basic of the constraints imposed by REST: Communication between client and server must be stateless. This means the state of the user should not be stored anywhere. You will look the other way in this particular case, however. I’ll cover the basics of each method anyway, because in real life, you have to compromise and you have to find a balance between purism and practicality. But don’t worry. I’ll go over an alternative design that will solve authentication and stay true to REST. Basic Auth with TSL Thanks to the fact that you’re basing REST on HTTP for the purpose of this book, the latter provides a basic authentication method that most of the languages can support. Keep in mind, though, that this method is aptly named, since it’s quite basic and works by sending the username and password unencrypted over HTTP. So the only way to make it secure is to use it with a secured connection over HTTPS (HTTP + TSL). This authentication method works as follows (see Figure 2-7): 58 Chapter 2 API Design Best Practices 1. First, a client makes a request for a resource without any special header. 2. The server responds with a 401 unauthorized response, and within it, a WWW-Authenticate header, specifying the method to use (Basic or Digest) and the realm name. 3. The client then sends the same request but adds the Authorization header, with the string USERNAME:PASSWORD encoded in base 64. On the server side, there needs to be some code to decode the authentication string and load the user data from the session storage used (normally a database). Figure 2-7. The steps between client and server on Basic Auth Aside from the fact that this approach is one of the many that will break the nonstateless constraint, it’s easy and fast to implement. Note When using this method, if the password for a logged in user is reset, then the login data sent on the request becomes old and the current session is terminated. 59 Chapter 2 API Design Best Practices Digest Auth This method is an improvement over the previous one, in the sense that it adds an extra layer of security by encrypting the login information. The communication with the server works the same way, by sending the same headers back and forth. With this methodology, upon receiving a request for a protected resource, the server will respond with a WWW-Authenticate header and some specific parameters. Here are some of the most interesting: • Nounce: A uniquely generated string. This string needs to be unique on every 401 response. • Opaque: A string returned by the server that has to be sent back by the client unaltered • Qop: Even though optional, this parameter should be sent to specify the quality of protection needed (more than one token can be sent in this value). Sending auth back would imply a simple authentication, whereas sending auth-int implies authentication with integrity check. • Algorithm: This string specifies the algorithm used to calculate the checksum response from the client. If not present, then MD5 should be assumed. For the full list of parameters and implementation details, please refer to the RFC.16 Here is a list of some of the most interesting ones: • Username: The unencrypted username. • URI: The URI you’re trying to access. • Response: The encrypted portion of the response. This proves that you are who you say you are. • Qop: If present, it should be one of the supported values sent by the server. 16See https://www.ietf.org/rfc/rfc2617.txt. 60 Chapter 2 API Design Best Practices To calculate the response, the following logic needs to be applied: MD5(HA1:STRING:HA2) Those values for HA1 are calculated as follows: • If no algorithm is specified on the response, then MD5(username:realm:password) should be used. • If the algorithm is MD5-less, then it should be MD5(MD5(username:realm:password):nonce:cnonce) Those values for HA2 are calculated as follows: • If qop is auth, then MD5(method:digestURI) should be used. • If qop is auth-int, then MD5(method:digestURI:MD5(entityBody)) should be used. Finally, the response will be as follows: MD5(HA1:nonce:nonceCount:clientNonce:HA2) //for the case when "qop" is "auth" or "auth-int" MD5(HA1:nonce:HA2) //when "qop" is unspecified. The main issue with this method is that the encryption used is based on MD5, and in 2004 it was proven that this algorithm is not collision-resistant, which basically means a man-in-the-middle attack would make it possible for an attacker to get the necessary information and generate a set of valid credentials. A possible improvement over this method, just like with its “Basic” brother, would be adding TSL; this would definitely help make it more secure. 61 Chapter 2 API Design Best Practices OAuth 1.0a OAuth 1.0a is the most secure of the four nonstateless methodologies described in this section. The process is a bit more tedious than the ones described earlier (see Figure 2-8), but the trade-off here is a significantly increased level of security. In this case, the service provider has to allow the developer of the client app to register the app on the provider’s web site. By doing so, the developer obtains a consumer key (a unique identifying key for his application) and the consumer secret. Once that process is done, the following steps are required: 1. The client app needs a request token. The purpose is to receive the user’s approval and then request an access token. To get the request token, a specific URL must be provided by the server; in this step, the consumer key and the consumer secret are used. 2. Once the request token is obtained, the client must make a request using the token on a specific server URL (i.e., http:// provider.com/oauth/authorize) to get authorization from the end user. 3. After authorization from the user is given, then the client app makes a request to the provider for an access token and a token secret key. 4. Once the access token and secret token are obtained, the client app is able to request protected resources for the provider on behalf of the user by signing each request. 62 Chapter 2 API Design Best Practices Figure 2-8. The interaction between client and server 63 Chapter 2 API Design Best Practices For more details on how this method works, please refer to the complete documentation.17 OAuth 2.0 OAuth 2.0 is meant to be the evolution of OAuth 1.0a; it focuses on client developer simplicity. The main problem with implementations of systems that worked with OAuth 1.0 was the complexity implied in the last step: signing every request. Due to its complexity, the last step is the key weak point of the algorithm: if either the client or server makes a tiny mistake, then the requests will not validate. Even when the same aspect made it the only methodology that didn’t need to work on top of SSL (or TSL), this benefit wasn’t enough. OAuth 2.0 tries to simplify the last step by making some key changes, mainly: • It relies on SSL (or TSL) to ensure that the information sent back and forth is encrypted. • Signatures are not required for requests after the token has been generated. To summarize, this version of OAuth tries to simplify the complexity introduced by OAuth 1.0, while sacrificing security at the same time (by relying on TSL to ensure data encryption). It is the preferred method over OAuth 1.0 if the devices you’re dealing with have support for TSL (computers, mobile devices, etc.); otherwise, you might want to consider using other options. A Stateless Alternative As you’ve seen, the alternatives you have when it comes to implementing a security protocol to allow users to sign into a RESTful API are not stateless, and even though you should be prepared to make that commitment to gain the benefits of tried and tested ways of securing your application, there is a fully REST compatible way of doing it as well. If you go back to Chapter 1, the stateless constraints basically imply that any and all states of the communication between client and server should be included on every request made by the client. This of course includes the user information, so if you want to have stateless authentication, then you need to include that in your requests as well. 17See http://oauth.net/core/1.0a/. 64 Chapter 2 API Design Best Practices If you want to ensure the authenticity of each request, you can borrow the signature step of OAuth 1.0a and apply it on every request by using a pre-established secret key between the client and the server and a MAC (Message Authentication Code) algorithm to do the signing (see Figure 2-9). As you’re keeping it stateless, the information required to generate the MAC needs to also be sent as part of the request, so the server can re-create the result and corroborate its validity. This approach has some clear advantages in our case, mainly: • It’s simpler than both OAuth 1.0a and OAuth 2.0. • Zero storage is needed, since any and all required information to validate the encryption needs to be sent on every request. Figure 2-9. How the MAC signing process works 65 Chapter 2 API Design Best Practices Scalability Last but certainly not least is scalability. Scalability is usually an underestimated aspect of API design, mainly because it’s quite difficult to fully understand and predict the reach one API will have before it launches. It might be easier to estimate this if the team has previous experience with similar projects (e.g., Google has probably gotten quite good at calculating their scalability for new APIs before launch day), but if it’s their first one, then it might not be as easy. A good API should be able to scale—that means it should be able to handle as much traffic as it gets without compromising its performance. But it also means it should not spend resources if they’re not needed. This is not only a reflection of the hardware on which the API resides (although that is an important aspect), but also a reflection of the underlying architecture of that API. Over the years, the classic monolithic design in software architecture has been migrating into a fully distributed one, so splitting the API into different modules that interact with each other makes sense. This provides the flexibility needed to not only scale up or down the resources that are affected but to also provide fault tolerance and help developers maintain cleaner code bases among other advantages. The following image (Figure 2-10) shows a standard monolithic design, having your app inside one server, living like one single entity. 66 Chapter 2 API Design Best Practices Figure 2-10. Simple diagram of a monolithic architecture 67 Chapter 2 API Design Best Practices In Figure 2-11 you see a distributed design. If compared with Figure 2-10, you can see where the advantages come from (better resource usage, fault tolerance, easier to scale up or down, etc.). Figure 2-11. A diagram showing an example of a distributed architecture 68 """