Summary: If you are using the rspec_api_documentation gem to test and document your API, you can use this code to test that you have a spec for each of your routes. Check out the gist here: http://gist.github.com/kayline/8868438
If you’re building an app with a public API, accurate and clear documentation is a blessing for both you and your third-party developers. The rspec_api_documentation gem is a nice solution that allows you to test your API and generate your docs in one fell swoop. You know your docs are up to date, because the tests that generate them are green. But what if an endpoint has no API test? Can we write a test to tell us when another test is missing?
As it turns out, yes! This post will explain how to create an Rspec test that fails if any of your routes do not have matching API documentation tests.
A Test Coverage Test
The test file itself lives in the same folder as your API specs. This is important. You must run your API specs at the same time that you run this test, or it won’t work properly. We put this code in spec/api/route_spec.rb.
require 'spec_helper'
require 'support/route_check'
it "tests all routes" do
Rails.application.reload_routes!
route_check =
RouteCheck.new(
routes: Rails.application.routes.routes,
world: RSpec::world)
route_check.missing_docs.should == Set.new
end
endThe test is implemented using a RouteCheck class, which takes in a list of routes and an Rspec test world. Where do these inputs come from?
Rails provides an API for getting all defined routes within an application, although the call is a little odd. Rails.application.routes returns a non-enumerable RouteSet object, on which we then call .routes again to get an array of routes. (We call Rails.application.reload_routes! first to ensure that all our defined routes have been loaded into the application.)
Getting the test world is fairly straightforward: just call Rspec::world. The trick here is that the world is loaded when the test is run, and its contents depend on which tests are being run. If you run a test in isolation, the world will only contain that single test. More to the point, if you are not running your API specs then Rspec::world won’t contain them and RouteCheck won’t be able to find them. That’s why this test lives in the API specs folder – it only works if the API specs are also running.
Test-Driving the Test Coverage Test
In order to implement our API coverage test, we need to define a RouteCheck class with the necessary behavior. In the spirit of recursive TDD, we wrote a series of Rspec tests to define how RouteCheck should behave. The following lives in spec/lib/route_check_spec.rb.
describe RouteCheck do
let!(:application_routes)
{ ActionDispatch::Routing::RouteSet.new }
let!(:route)
{ route_builder(
'GET',
'projects/lebowski',
'projects#lebowski',
application_routes) }
let!(:devise_route)
{ route_builder(
'POST',
'devise/users',
'users#bloop',
application_routes)}
let!(:internal_route)
{ route_builder(
'GET',
'/assets',
'rails#get_info',
application_routes) }
describe '#filtered_routes' do
let(:routes) { RouteCheck.new(routes: application_routes.routes) }
let(:filtered_routes) { routes.filtered_routes }
it 'includes all non-filtered routes' do
filtered_routes.should == [route]
end
end
describe '#api_specs' do
subject(:route_check) { RouteCheck.new(world: world) }
let(:configuration) { RSpec::Core::Configuration.new }
let(:world) { RSpec::Core::World.new(configuration) }
describe 'returns routes for tests found in the world' do
context "when there are no examples" do
it "is empty" do
route_check.api_specs.should be_empty
end
end
context "when there are non-API examples" do
let(:regular_group)
{ RSpec::Core::ExampleGroup.describe("regular group") }
let(:action_context)
{ resource_group.context(
"/path/to/api",
{ :api_doc_dsl => :endpoint,
:method => "METHOD",
:route => "/path/to/api" }
) }
let(:resource_group) do
RSpec::Core::ExampleGroup.
describe("resource group",{:api_doc_dsl=>:resource}) do
context "something different"
end
end
before do
world.register(regular_group)
world.register(resource_group)
action_context.register
end
it "returns only API test groups" do
route_check.api_specs.should == [action_context]
end
end
end
end
describe '#missing_docs' do
subject(:route_check)
{ RouteCheck.new(routes: application_routes.routes, world: world) }
let(:configuration)
{ RSpec::Core::Configuration.new }
let(:world)
{ RSpec::Core::World.new(configuration) }
let(:wrapped_route)
{ ActionDispatch::Routing::RouteWrapper.new(route) }
let(:formatted_route)
{ ::Route.new(
wrapped_route.verb.downcase,
wrapped_route.path[//[^( ]+/]) }
it 'detects routes for which no api test exists' do
route_check.missing_docs.should == [formatted_route].to_set
end
it 'does not return routes for which an api spec exists' do
group = RSpec::Core::ExampleGroup.
describe("resource group", {:api_doc_dsl => :resource}) do
context("/path/to/api",
{:api_doc_dsl => :endpoint,
:method => :get,
:route => '/projects/lebowski'})
end
world.register(group)
route_check.missing_docs.should be_empty
end
end
def route_builder(method, path, action, route_set)
scope = {:path_names=>{:new=>"new", :edit=>"edit"}}
path = path
name = path.split("/").last
options = {:via => method, :to => action, :anchor => true, :as => name}
mapping = ActionDispatch::Routing::Mapper::Mapping.
new(route_set, scope, path, options)
app, conditions, requirements, defaults, as, anchor = mapping.to_route
route_set.add_route(app, conditions, requirements, defaults, as, anchor)
end
endWe wrote tests for three methods on the RouteCheck class. The first, filtered_routes, should return the list of all routes for which we want matching API specs. This should not include internal Rails routes or routes without an HTTP method, and should allow us to filter out other routes we may not want to test (for example, routes generated by the Devise gem).
The second method, api_specs, should return a list of all currently defined rspec_api_documentation tests. It should not include non-API tests.
Finally, the missing_docs method should use the outputs from filtered_routes and api_specs to return all of the chosen routes that do not have matching tests.
The most complex part of these tests is creating new routes and tests in the test world without defining them in the app. We started by creating an empty RouteSet object and an empty Rspec world. In order to add routes to the RouteSet we created a route_builder method, which handles the creation of necessary intermediary Rails objects.
To add a new example group to the Rspec world we call RSpec::Core::ExampleGroup.describe() and pass in the test description and an options hash. Inside the example group we define a context, which is an actual test. In order to indicate that we want a certain test to be considered an API test, we add the api_doc_dsl key to the metadata of both the example group and the test. We did not find a way to organically add an API documentation test to the Rspec world, other than hard-setting the metadata key.
Implementation
Now that we have some solid tests, we are ready to implement the RouteCheck class. We put this class in spec/support/route_check.rb.
class RouteCheck
attr_reader :routes, :world
def initialize(routes: nil, world: nil)
@routes = routes
@world = world
end
def filtered_routes
collect_routes do |route|
next if route.internal?
next if route.verb.blank?
next if route.controller =~ /^devise.*/
next if route.controller =~ /^docs.*/
next if route.controller == "users"
next if route.controller == "authentication_jig"
next if route.controller == "invitations"
next if route.controller == "admin"
next if route.controller == "index"
route
end.compact
end
def api_specs
world.
example_groups.
map(&:descendants).
flatten.
reject{ |g| g.metadata.fetch(:api_doc_dsl, :not_found) == :not_found }.
reject{ |g| g.metadata.fetch(:method, :not_found) == :not_found }
end
def missing_docs
existing_routes = Set.new(matchable_routes(filtered_routes))
existing_route_specs = Set.new(matchable_specs(api_specs))
existing_routes - existing_route_specs
end
private
def matchable_routes(routes)
routes.collect do |r|
::Route.new(r.verb, r.path[//[^( ]+/])
end.compact
end
def matchable_specs(specs)
specs.map do |spec|
::Route.new(spec.metadata[:method], spec.metadata[:route])
end
end
def collect_routes
routes.collect do |route|
route = yield ActionDispatch::Routing::RouteWrapper.new(route)
end
end
end
class ::Route < Struct.new(:method, :path)
def eql? other
self.hash == other.hash
end
def == other
method.to_s.downcase == other.method.to_s.downcase and
path.downcase == other.path.downcase
end
def hash
method.to_s.downcase.hash + path.downcase.hash
end
endThe implementation of the filtered_routes method uses a private method collect_routes to wrap each route in a RouteWrapper object (a Rails class that provides simple accessor methods for route properties), and collect the results in a new array. The filtered_routes method itself passes a block to collect_routes that tells it to ignore any routes that meet certain conditions. The first two conditions filter out routes that are
a) internal to Rails
b) have no method (‘GET’, ‘POST’, ‘CREATE’, ‘DELETE’)
You will probably always want to include these two conditions in your filter. These rest of the filter conditions are project-dependent. We wrote this class for a project that was using the Devise gem for user management. Because Devise is an externally-maintained library with its own set of tests, we did not write API specs for Devise endpoints. We therefore tell filtered_routes to ignore any routes pointing to a Devise controller (/devise, users, or invitations). We also ignore a few routes that are not part of our public API. Which routes to filter is up to you, but keep in mind that the purpose of this code is to keep you honest about your API test coverage.
On to spec parsing! The RouteCheck class takes in an Rspec world and uses the api_specs method to extract each rspec_api_documentation test. Rspec has a pretty friendly API, so the implementation is relatively simple. We extract all the example groups and their descendants from the world, which gives us an array of rspec test objects. Each of these objects has a metadata hash, and the rspec_api_documentation gem helpfully adds an :api_doc_dsl key to that hash for all api_spec tests. By filtering our array of tests for those that have this key, we end up with an array of rspec_api_documentation test objects. However, this includes both actual tests and context blocks, so we filter again for objects that have a :method key in their metadata. Voila! We now have a test object for every API spec, and this object knows its method and path (via its metadata).
Now that we have an array of routes and an array of tests, we can call the missing_docs method to find the difference between the two. This method formats both the routes and the tests as ::Route objects with a custom equality function. It then subtracts the set of all tested routes from the set of all filtered routes. Any routes that have not been tested will be contained in the diff.
