🛠️ REST_GO: Comprehensive Collection of REST API Testing Tools & Services

Welcome to REST_GO, a repository that collects REST API Testing Tools and RESTful Services.

🎯 Purpose:

With the ever-growing importance of RESTful web services, effective testing methods are paramount. Our repository aims to serve as a one-stop resource for both researchers and practitioners in the realm of REST API testing. By consolidating the most valuable tools and services, we hope to advance the state of the art and ease the tasks of developers and testers alike.

📜 Cite Us:

If you find the resources here beneficial, we'd appreciate it if you consider citing our research paper, titled "Automated Test Generation for REST APIs: No Time to Rest Yet".

Bibtex Citation Here

Tools

We currently have 11 open source REST API testing projects:

1. EvoMaster: Evolutionary Algorithm and Dynamic Program Analysis approach
2. RESTler: Identifies producer-consumer dependencies among request types using OpenAPI definition
3. RestTestGen: Generates valid call sequences by constructing and traversing an Operation Dependency Graph (ODG) based on API dependencies
4. RESTest: Supports inter-parameter dependency specification and analysis using constraint-based testing
5. bBOXRT: Robustness testing tool for REST services, generating invalid inputs combined with valid parameters based on a service description document
6. Schemathesis: Property-based testing approach
7. Tcases: Models input space using operation's input and output for comprehensive testing
8. Dredd: Validates responses based on status codes, headers, and body payloads using sample values from the specification and dummy values
9. APIFuzzer: Employs fuzzing with random values and mutations
10. RestCT: Combinatorial testing-based approach
11. Morest: Dynamically updates RESTful-service Property Graph (RPG) to model API and object schema information.

Given the frequent updates to most of these tools, I recommend visiting their official websites for the latest installation instructions.

Services

We have identified 20 open-source Java-based RESTful services:

1. Corona-Warn-App Verification Server: This project has a rate limit (1000 requests per hour).
2. ERC-20 RESTful service: This project crashes due to an update in the external dependency (web3j). Since this project is no longer maintained, we cannot use it anymore.
3. Features Model MicroService
4. Genome Nexus
5. LanguageTool
6. Simple internet-market
7. NCS
8. News
9. OCVN: Currently, we are having an authentication issue for some operations.
10. Person Controller
11. Problem & Project Controller: Functional Issue - Cannot obtain code resource
12. Project Tracking System
13. proxyprint-kitchen: Currently, we are having an authentication issue for some operations.
14. RESTful web service study: This service is no longer accessible.
15. REST Countries
16. SCS
17. Scout API: This service is no longer accessible.
18. Spring Boot Actuator: Currently, we are facing a broken dependency issue.
19. Spring Batch REST: Functional Issue - Cannot obtain job resource
20. User Management Microservice

We've collected all services' source codes in services directory and all specifications in specifications directory. To use the services, please install the services (check setup.sh as a reference) and dependencies. Then, you can use the python script to run the services. Command Line Arguments:

• SERVICE_NAME: The name of the service you want to run. possible names are: features-service, languagetool, ncs, news, ocvn, proxyprint, restcountries, scout-api, scs, erc20-rest-service, genome-nexus, person-controller, problem-controller, rest-study, spring-batch-rest, spring-boot-sample-app, user-management, cwa-verification, market, project-tracking-system
• COVERAGE_PORT: The port for JaCoCo (Java Code Coverage Library).
• WHITEBOX/BLACKBOX: The approach (either "whitebox" or "blackbox").

python3 run_service.py {SERVICE_NAME} {COVERAGE_PORT} {WHITEBOX/BLACKBOX}

Measuring Code Coverage

For a precise measurement of code coverage, follow these steps:

1. Set Up the Tools: Download the Jacoco Agent and Jacoco CLI. These tools will aid in the measurement process.
2. Run the Project with Jacoco Agent: Launch each project with the following option, replacing {COVERAGE_PORT} with your chosen port number:

-javaagent:org.jacoco.agent-0.8.7-runtime.jar=includes=*,output=tcpserver,port={COVERAGE_PORT},address=*,dumponexit=true -Dfile.encoding=UTF-8

3. Run the Coverage Script: Execute the get_cov.sh script:

sh get_cov.sh {COVERAGE_PORT}

This script will produce coverage files at 10-minute intervals over the span of an hour, resulting in files named:

10 minute: jacoco_{COVERAGE_PORT}_1.exec
20 minute: jacoco_{COVERAGE_PORT}_2.exec
30 minute: jacoco_{COVERAGE_PORT}_3.exec
40 minute: jacoco_{COVERAGE_PORT}_4.exec
50 minute: jacoco_{COVERAGE_PORT}_5.exec
60 minute: jacoco_{COVERAGE_PORT}_6.exec

Generating Coverage and Error Report

After having the coverage result, run this command to generate the coverage and error report:

python3 report.py {COVERAGE_PORT} {SOURCE_CODE_LOCATION}

Please note that you need to have all the executable Jacoco files generated in the previous step in the same directory. It will generate a report directory with error.json that contains error report and res.csv that contains coverage report.

Add authorization header

Some APIs can have an authorization header to increase the API request limit. We recommend to use mitmproxy to add the authorization header.

You need to add your token in authToken.py (Replce TOKEN_HERE with an appropriate token) and run the following command with the service URL and proxy URL. Mitmproxy accepts the request and forwards it to the specified upstream server.

mitmproxy --mode reverse:SERVICE_URL -p PROXY_PORT_NUMBER -s authToken.py

Proof of Concepts

We provide two proof-of-concept prototypes that help to find example value, inter-parameter dependency, and linked response parameter for each request parameter. proof-of-concept1.py takes parameter description and parameter names in the operation and produce example values and inter-parameter dependency. proof-of-concept2.py takes request parameter names and response parameter names in the specification and produce request parameter and response parameter pairs. Each request parameter name has three response parameter names that are top three similar names.

python3 tools/proof-of-concept1.py {parameter description} {parameter names}
python3 tools/proof-of-concept2.py {request parameter names} {response parameter names}

However, instead of this proof of concept tools, we highly recommend to read Enhancing REST API Testing with NLP Techniques and use NLP2REST as it is more advanced technique.