REST API Fuzzer And Destructive Testing Software For OpenAPI Endpoints

REST API fuzzer and destructive testing software. Run hundreds of self-healing API checks inside minutes with no coding effort!

Complete: checks are generated routinely based mostly on a big quantity situations and canopy each discipline and header
Clever: checks are generated based mostly on knowledge sorts and constraints; every Fuzzer have particular expectations relying on the state of affairs underneath take a look at
Extremely Configurable: excessive quantity of customization: you’ll be able to exclude particular Fuzzers, HTTP response codes, present enterprise context and much more
Self-Therapeutic: as checks are generated, any OpenAPI spec change is picked up routinely
Easy to Study: flat studying curve, with intuitive configuration and syntax
Quick: computerized course of for write, run and report checks which covers hundreds of situations inside minutes

By utilizing a easy and minimal syntax, with a flat studying curve, CATS (Contract Auto-generated Tests for Swagger) allows you to generate hundreds of API checks inside minutes with no coding effort.
All checks are generated, run and reported routinely based mostly on a pre-defined set of 89 Fuzzers.
The Fuzzers cowl a variety of enter knowledge from absolutely random giant Unicode values to nicely crafted, context dependant values based mostly on the request knowledge sorts and constraints.
Much more, you’ll be able to leverage the truth that CATS generates request payloads dynamically and write easy end-to-end useful checks.

Please test the Slicing Methods part for making CATS run quick and complete in the identical time.

This can be a listing of articles with step-by-step guides on learn how to use CATS:

Homebrew

> brew faucet endava/faucet
> brew set up cats

Guide

CATS is bundled each as an executable JAR or a local binary. The native binaries don’t want Java put in.

After downloading your OS native binary, you’ll be able to add it in classpath so to execute it as another command line software:

sudo cp cats /usr/native/bin/cats

You can too get autocomplete by downloading the cats_autocomplete script and do:

To get persistent autocomplete, add the above line in ~/.zshrc or ./bashrc, however ensure you put the absolutely certified path for the cats_autocomplete script.

You can too test the cats_autocomplete supply for various setup.

There isn’t any native binary for Home windows, however you need to use the uberjar model. This requires Java 11+ to be put in.

You possibly can run it as java -jar cats.jar.

Head to the releases web page to obtain the newest variations: https://github.com/Endava/cats/releases.

Construct

You possibly can construct CATS from sources on you native field. You want Java 11+. Maven is already bundled.

Earlier than operating the primary construct, please ensure you do a ./mvnw clear. CATS makes use of a fork okay OKHttpClient which can set up regionally
underneath the 4.9.1-CATS model, so don’t be concerned about overriding the official variations.

You should utilize the next Maven command to construct the undertaking:

./mvnw bundle -Dquarkus.bundle.sort=uber-jar

cp goal/

You’ll find yourself with a cats.jar within the goal folder. You possibly can run it wih java -jar cats.jar ....

You can too construct native photos utilizing a GraalVM Java model.

./mvnw bundle -Pnative

Notice: You’ll need to configure Maven with a Github PAT with read-packages scope to get some dependencies for the construct.

Notes on Unit Checks

You might even see some ERROR log messages whereas operating the Unit Checks. These are anticipated behaviour for testing the destructive situations of the Fuzzers.

Blackbox mode

Blackbox mode signifies that CATS would not want any particular context. You simply want to offer the service URL, the OpenAPI spec and likely authentication headers.

> cats --contract=openapy.yaml --server=http://localhost:8080 --headers=headers.yml --blackbox

In blackbox mode CATS will solely report ERRORs if the acquired HTTP response code is a 5XX.
Every other mismatch between what the Fuzzer expects vs what the service returns (for instance service returns 400 and repair returns 200) will likely be ignored.

The blackbox mode is just like a smoke take a look at. It is going to shortly inform you if the applying has main bugs that should be addressed instantly.

Context mode

The actual energy of CATS depends on operating it in a non-blackbox mode additionally referred to as context mode.
Every Fuzzer has an anticipated HTTP response code based mostly on the state of affairs underneath take a look at and also will test if the response is matching the schema outlined within the OpenAPI spec particular to that response code.
It will let you tweak both your OpenAPI spec or service behaviour in an effort to create good high quality APIs and documentation and likewise to keep away from potential critical bugs.

Working CATS in context mode often implies offering it a –refData file with useful resource identifiers particular to the enterprise logic.
CATS can not create knowledge by itself (but), so it is vital that any request discipline or question param that requires pre-existence of these entities/assets to be created upfront and added to the reference knowledge file.

> cats --contract=openapy.yaml --server=http://localhost:8080 --headers=headers.yml --refData=referenceData.yml

Notes on skipped Checks

You might discover a big variety of checks marked as skipped. CATS will attempt to apply all Fuzzers to all fields, however this isn’t at all times potential.
For instance the BooleanFieldsFuzzer can’t be utilized to String fields. This is the reason that take a look at try will likely be marked as skipped.
It was an intentional determination to additionally report the skipped checks in an effort to present that CATS really tries all of the Fuzzers on all of the fields/paths/endpoints.

Moreover, CATS assist quite a bit extra arguments that lets you limit the variety of fuzzers, present timeouts, restrict the variety of requests per minute and so forth.

CATS generates checks based mostly on configured Fuzzers. Every Fuzzer has a selected state of affairs and a selected anticipated consequence.
The CATS engine will run the state of affairs, get the consequence from the service and match it with the Fuzzer anticipated consequence.
Relying on the matching end result, CATS will report as follows:

INFO/SUCCESS is predicted and documented behaviour. No want for motion.
WARN is predicted however undocumented behaviour or some misalignment between the contract and the service. It will ideally be actioned.
ERROR is irregular/sudden behaviour. This should be actioned.

CATS will iterate by all endpoints, all HTTP strategies and all of the related requests our bodies and parameters (together with a number of mixtures when coping with oneOf/anyOf parts) and fuzz their values contemplating their outlined knowledge sort and constraints.
The precise fuzzing depends upon the particular Fuzzer executed. Please see the listing of fuzzers and their behaviour.
There are additionally variations on how the fuzzing works relying on the HTTP methodology:

for strategies with request our bodies like POST, PUT the fuzzing will likely be utilized on the request physique knowledge fashions stage
for strategies with out request our bodies like GET, DELETE the fuzzing will likely be utilized on the URL parameters stage

Which means that for strategies with request our bodies (POST,PUT) which have additionally URL/path parameters, you want to provide the path parameters through urlParams or the referenceData file as failure to take action will end in Unlawful character in path at index ... errors.

HTML_JS

HTML_JS is the default report produced by CATS. The execution report in positioned a folder referred to as cats-report/TIMESTAMP or cats-report relying on the --timestampReports argument. The folder will likely be created inside the present folder (if it would not exist) and for every run a brand new subfolder will likely be
created with the TIMESTAMP worth when the run began. This lets you have a historical past of the runs. The report itself is within the index.html file, the place you’ll be able to:

filter take a look at runs based mostly on the consequence: All, Success, Warn and Error
filter based mostly on the Fuzzer so to solely see the runs for that particular Fuzzer
see abstract with all of the checks with their corresponding path towards they had been run, and the consequence
have capability to click on on any checks and get particulars in regards to the Situation being executed, Anticipated Consequence, Precise consequence in addition to request/response particulars

Together with the abstract from index.html every particular person take a look at may have a selected TestXXX.html web page with extra particulars, in addition to a json model of the take a look at which will be latter replayed utilizing > cats replay TestXXX.json.

Understanding the Consequence Motive values:

Sudden Exception – reported as error; this would possibly point out a potential bug within the service or a nook case that isn’t dealt with appropriately by CATS
Not Matching Response Schema – reported as a warn; this means that the service returns an anticipated response code and a response physique, however the response physique doesn’t match the schema outlined within the contract
Undocumented Response Code – reported as a warn; this means that the service returns an anticipated response code, however the response code will not be documented within the contract
Sudden Response Code – reported as an error; this means a potential bug within the service – the response code is documented, however will not be anticipated for this state of affairs
Sudden Behaviour – reported as an error; this means a potential bug within the service – the response code is neither documented nor anticipated for this state of affairs
Not Discovered – reported as an error in an effort to drive offering extra context; this means that CATS wants further enterprise context in an effort to run efficiently – you are able to do this utilizing the --refData and/or --urlParams arguments

That is the abstract web page:

And that is what you get if you click on on a selected take a look at:

HTML_ONLY

This format is comparable with HTML_JS, however you can’t do any filtering or sorting.

JUNIT

CATS additionally helps JUNIT output. The output will likely be a single testsuite that may incorporate all checks grouped by Fuzzer identify.
Because the JUNIT format doesn’t have the idea of warning the next mapping is used:

CATS error is reported as JUNIT error
JUNIT failure will not be used in any respect
CATS warn is reported as JUNIT skipped
CATS skipped is reported as JUNIT disabled

The JUNIT report is written as junit.xml within the cats-report folder. Particular person checks, each as .html and .json will even be created.

CATS has a big variety of Fuzzers. At the moment, 89 and rising. Among the Fuzzers are executing a number of checks for each given discipline throughout the request.
For instance the ControlCharsOnlyInFieldsFuzzer has 63 management chars values that will likely be tried for every request discipline. If a request has 15 fields for instance, it will end in 1020 checks.
Contemplating that there are further Fuzzers with the identical magnitude of checks being generated, you’ll be able to simply get to 20k checks being executed on a typical run. It will end in enormous stories and future occasions (i.e. minutes, relatively than seconds).

Beneath are some really helpful methods on how one can separate the checks in chunks which will be executed as levels in a deployment pipeline, one after the opposite.

Cut up by Endpoints

You should utilize the --paths=PATH argument to run CATS sequentially for every path.

Cut up by Fuzzer Class

You should utilize the --checkXXX arguments to run CATS solely with particular Fuzzers like: --checkHttp, -checkFields, and so forth.

Cut up by Fuzzer Sort

You should utilize numerous arguments like --fuzzers=Fuzzer1,Fuzzer2 or -skipFuzzers=Fuzzer1,Fuzzer2 to both embody or exclude particular Fuzzers.
For instance, you’ll be able to run all Fuzzers aside from the ControlChars and Whitespaces ones like this: --skipFuzzers=ControlChars,Whitesspaces. It will skip all Fuzzers containing these strings of their identify.
After, you’ll be able to create an extra run solely with these Fuzzers: --fuzzers=ControlChars,Whitespaces.

These are just a few suggestions on how one can break up the kinds of checks circumstances. Relying on how advanced your API is, you would possibly go along with a mixture of the above or with much more granular splits.

Please be aware that as a result of the truth that ControlChars, Emojis and Whitespaces generate enormous variety of checks even for small OpenAPI contracts, they’re disabled by default.
You possibly can allow them utilizing the --includeControlChars, --includeWhitespaces and/or --includeEmojis arguments.
The advice is to run them in separate runs so that you simply get manageable stories and optimum operating occasions.

By default, CATS will report WARNs and ERRORs in response to the particular behaviour of every Fuzzer. There are circumstances although if you would possibly need to focus solely on vital bugs.
You should utilize the --ignoreResponseXXX arguments to produce a listing of response codes, response sizes, phrase counts, line counts or response physique regexes that ought to be ignored as points (overriding the Fuzzer behaviour) and report these circumstances as success as a substitute or WARN or ERROR.
For instance, if you’d like CATS to report ERRORs solely when there may be an Exception or the service returns a 500, you need to use this: --ignoreResultCodes="2xx,4xx".

You can too select to disregard checks finished by the Fuzzers. By default, every Fuzzer has an anticipated response code, based mostly on the state of affairs underneath take a look at and can report and WARN the service returns the anticipated response code,
however the response code will not be documented contained in the contract.
You can also make CATS ignore the undocumented response code checks (i.e. checking anticipated response code contained in the contract) utilizing the --ignoreResponseCodeUndocumentedCheck argument. CATS with now report these circumstances as SUCCESS as a substitute of WARN.

Moreover, you can too select to disregard the response physique checks. By default, on high of checking the anticipated response code, every Fuzzer will test if the response physique matches what’s outlined within the contract and can report an WARN if not matching.
You can also make CATS ignore the response physique checks utilizing the --ingoreResponseBodyCheck argument. CATS with now report these circumstances as SUCCESS as a substitute of WARN.

When CATS runs, for every take a look at, it is going to export each an HTML file that will likely be linked within the ultimate report and particular person JSON information. The JSON information can be utilized to replay that take a look at.
When replaying a take a look at (or a listing of checks), CATS will not produce any report. The output will likely be solely accessible within the console.
That is helpful if you need to see the precise behaviour of the particular take a look at or connect it in a bug report for instance.

The syntax for replaying checks is the next:

> cats replay "Test1,Test233,Test15.json,dir/Test19.json"

Some notes on the above instance:

take a look at names will be separated by comma ,
when you present a json extension to a take a look at identify, that file will likely be search as a path i.e. it is going to seek for Test15.json within the present folder and Test19.json within the dir folder
when you do not present a json extension to a take a look at identify, it is going to seek for that take a look at within the cats-report folder i.e. cats-report/Test1.json and cats-report/Test233.json

To listing all accessible instructions, run:

All accessible subcommands are listed under:

> cats assist or cats -h will listing all accessible choices
> cats listing --fuzzers will listing all the prevailing fuzzers, grouped on classes
> cats listing --fieldsFuzzingStrategy will listing all of the accessible fields fuzzing methods
> cats listing --paths --contract=CONTRACT will listing all of the paths accessible throughout the contract
> cats replay "test1,test2" will replay the given checks test1 and test2
> cats fuzz will fuzz based mostly on a given request template, relatively than an OpenAPI contract
> cats run will run useful and focused safety checks written within the CATS YAML format
> cats lint will run OpenAPI contract linters, additionally referred to as ContractInfoFuzzers

--contract=LOCATION_OF_THE_CONTRACT provides the placement of the OpenApi or Swagger contract.
--server=URL provides the URL of the service implementing the contract.
--basicauth=USR:PWD provides a username:password pair, in case the service makes use of fundamental auth.
--fuzzers=LIST_OF_FUZZERS provides a comma separated listing of fuzzers. The provided listing of Fuzzers will be partial names, not full Fuzzer names. CATS which test for all Fuzzers containing the provided strings. If the argument will not be provided, all fuzzers will likely be run.
--log=PACKAGE:LEVEL can configure customized log stage for a given bundle. You possibly can present a comma separated listing of packages and ranges. That is useful if you need to see full HTTP site visitors: --log=org.apache.http.wire:debug or suppress CATS logging: --log=com.endava.cats:warn
--paths=PATH_LIST provides a comma separated listing of OpenApi paths to be examined. If no path is provided, all paths will likely be thought of.
--skipPaths=PATH_LIST a comma separated listing of paths to disregard. If no path is provided, no path will likely be ignored
--fieldsFuzzingStrategy=STRATEGY specifies which technique will likely be used for discipline fuzzing. Out there methods are ONEBYONE, SIZE and POWERSET. Extra info on discipline fuzzing will be discovered within the sections under.
--maxFieldsToRemove=NUMBER specifies the utmost variety of fields to be eliminated when utilizing the SIZE fields fuzzing technique.
--refData=FILE specifies the file containing static reference knowledge which should be fastened in an effort to have legitimate enterprise requests. This can be a YAML file. It’s defined additional within the sections under.
--headers=FILE specifies a file containing headers that will likely be added when sending payloads to the endpoints. You should utilize this selection so as to add oauth/JWT tokens for instance.
--edgeSpacesStrategy=STRATEGY specifies learn how to anticipate the server to behave when sending trailing and prefix areas inside fields. Attainable values are trimAndValidate and validateAndTrim.
--sanitizationStrategy=STRATEGY specifies learn how to anticipate the server to behave when sending Unicode Management Chars and Unicode Different Symbols throughout the fields. Attainable values are sanitizeAndValidate and validateAndSanitize
--urlParams A comma separated listing of ‘identify:worth’ pairs of parameters to get replaced contained in the URLs. That is helpful when you will have static parameters in URLs (like ‘model’ for instance).
--functionalFuzzerFile a file utilized by the FunctionalFuzzer that will likely be used to create user-supplied payloads.
--skipFuzzers=LIST_OF_FIZZERs a comma separated listing of fuzzers that will likely be skipped for all paths. You possibly can both present full Fuzzer names (for instance: --skippedFuzzers=VeryLargeStringsFuzzer) or partial Fuzzer names (for instance: --skipFuzzers=VeryLarge). CATS will test if the Fuzzer names incorporates the string you present within the arguments worth.
--skipFields=field1,field2#subField1 a comma separated listing of fields that will likely be skipped by substitute Fuzzers like EmptyStringsInFields, NullValuesInFields, and so forth.
--httpMethods=PUT,POST,and so forth a comma separated listing of HTTP strategies that will likely be used to filter which http strategies will likely be executed for every path throughout the contract
--securityFuzzerFile A file utilized by the SecurityFuzzer that will likely be used to inject particular strings in an effort to exploit potential vulnerabilities
--printExecutionStatistics If provided (no worth wanted), prints a abstract of execution occasions for every endpoint and HTTP methodology. By default it will print a abstract for every endpoint: max, min and common. In order for you detailed stories it’s essential to provide --printExecutionStatistics=detailed
--timestampReports If provided (no worth wanted), it is going to output the report nonetheless contained in the cats-report folder, however in a sub-folder with the present timestamp
--reportFormat=FORMAT Specifies the format of the CATS report. Supported codecs: HTML_ONLY, HTML_JS or JUNIT. You should utilize HTML_ONLY if you’d like the report back to not comprise any Javascript. That is helpful in CI environments as a result of Javascript content material safety insurance policies. Default is HTML_JS which incorporates some sorting and filtering capabilities.
--useExamples If true (default worth when not provided) then CATS will use examples provided within the OpenAPI contact. If false CATS will rely solely on generated values
--checkFields If provided (no worth wanted), it is going to solely run the Subject Fuzzers
--checkHeaders If provided (no worth wanted), it is going to solely run the Header Fuzzers
--checkHttp If provided (no worth wanted), it is going to solely run the HTTP Fuzzers
--includeWhitespaces If provided (no worth wanted), it is going to embody the Whitespaces Fuzzers
--includeEmojis If provided (no worth wanted), it is going to embody the Emojis Fuzzers
--includeControlChars If provided (no worth wanted), it is going to embody the ControlChars Fuzzers
--includeContract If provided (no worth wanted), it is going to embody ContractInfoFuzzers
--sslKeystore Location of the JKS keystore holding certificates used when authenticating calls utilizing one-way or two-way SSL
--sslKeystorePwd The password of the sslKeystore
--sslKeyPwd The password of the non-public key from the sslKeystore
--proxyHost The proxy server’s host identify (if operating behind proxy)
--proxyPort The proxy server’s port quantity (if operating behind proxy)
--maxRequestsPerMinute Most variety of requests per minute; that is helpful when APIs have charge limiting carried out; default is 10000
--connectionTimeout Time interval in seconds which CATS ought to set up a reference to the server; default is 10 seconds
--writeTimeout Most time of inactivity in seconds between two knowledge packets when sending the request to the server; default is 10 seconds
--readTimeout Most time of inactivity in seconds between two knowledge packets when ready for the server’s response; default is 10 seconds
--dryRun If offered, it is going to simulate a run of the service with the provided configuration. The run will not produce a report, however will present what number of checks will likely be generated and run for every OpenAPI endpoint
--ignoreResponseCodes HTTP_CODES_LIST a comma separated listing of HTTP response codes that will likely be thought of as SUCCESS, even when the Fuzzer will sometimes report it as WARN or ERROR. You should utilize response code households as 2xx, 4xx, and so forth. If offered, all Contract Fuzzers will likely be skipped.
--ignoreResponseSize SIZE_LIST a comma separated listing of response sizes that will likely be thought of as SUCCESS, even when the Fuzzer will sometimes report it as WARN or ERROR
--ignoreResponseWords COUNT_LIST a comma separated listing of phrases depend within the response that will likely be thought of as SUCCESS, even when the Fuzzer will sometimes report it as WARN or ERROR
--ignoreResponseLines LINES_COUNT a comma separated listing of traces depend within the response that will likely be thought of as SUCCESS, even when the Fuzzer will sometimes report it as WARN or ERROR
--ignoreResponseRegex a REGEX that may match towards the response that will likely be thought of as SUCCESS, even when the Fuzzer will sometimes report it as WARN or ERROR
--tests TESTS_LIST a comma separated listing of executed checks in JSON format from the cats-report folder. In the event you provide the listing with out the .json extension CATS will search the take a look at within the cats-report folder
--ignoreResponseCodeUndocumentedCheck If provided (not worth wanted) it will not test if the response code acquired from the service matches the worth anticipated by the fuzzer and can return the take a look at consequence as SUCCESS as a substitute of WARN
--ignoreResponseBodyCheck If provided (not worth wanted) it will not test if the response physique acquired from the service matches the schema provided contained in the contract and can return the take a look at consequence as SUCCESS as a substitute of WARN
--blackbox If provided (no worth wanted) it is going to ignore all response codes aside from 5XX which will likely be returned as ERROR. That is just like --ignoreResponseCodes="2xx,4xx"
--contentType A customized mime sort if the OpenAPI spec makes use of content material sort negotiation versioning.
--outoput The trail the place the CATS report will likely be written. Default is cats-report within the present listing
--skipReportingForIgnoredCodes Skip reporting fully for the any ignored arguments offered in --ignoreResponseXXX

> cats --contract=my.yml --server=https://locathost:8080 --checkHeaders

It will run CATS towards http://localhost:8080 utilizing my.yml as an API spec and can solely run the HTTP headers Fuzzers.

To get a listing of fuzzers run cats listing --fuzzers. An inventory of all accessible fuzzers will likely be returned, together with a brief description for every.

There are a number of classes of Fuzzers accessible:

Subject Fuzzers which goal request physique fields or path parameters
Header Fuzzers which goal HTTP headers
HTTP Fuzzers which goal simply the interplay with the service (with out fuzzing fields or headers)

Extra checks which aren’t really utilizing any fuzzing, however leverage the CATS inner mannequin of operating the checks as Fuzzers:

ContractInfo Fuzzers which checks the contract for API good practices
Particular Fuzzers a particular class which want additional configuration and are targeted on extra advanced actions like useful circulate, safety testing or supplying your individual request templates, relatively than OpenAPI specs

Subject Fuzzers

CATS has at the moment 42 registered Subject Fuzzers:

BooleanFieldsFuzzer – iterate by every Boolean discipline and ship random strings within the focused discipline
DecimalFieldsLeftBoundaryFuzzer – iterate by every Quantity discipline (both float or double) and ship requests with exterior the vary values on the left aspect within the focused discipline
DecimalFieldsRightBoundaryFuzzer – iterate by every Quantity discipline (both float or double) and ship requests with exterior the vary values on the suitable aspect within the focused discipline
DecimalValuesInIntegerFieldsFuzzer – iterate by every Integer discipline and ship requests with decimal values within the focused discipline
EmptyStringValuesInFieldsFuzzer – iterate by every discipline and ship requests with empty String values within the focused discipline
ExtremeNegativeValueDecimalFieldsFuzzer – iterate by every Quantity discipline and ship requests with the bottom worth potential (-999999999999999999999999999999999999999999.99999999999 for no format, -3.4028235E38 for float and -1.7976931348623157E308 for double) within the focused discipline
ExtremeNegativeValueIntegerFieldsFuzzer – iterate by every Integer discipline and ship requests with the bottom worth potential (-9223372036854775808 for int32 and -18446744073709551616 for int64) within the focused discipline
ExtremePositiveValueDecimalFieldsFuzzer – iterate by every Quantity discipline and ship requests with the very best worth potential (999999999999999999999999999999999999999999.99999999999 for no format, 3.4028235E38 for float and 1.7976931348623157E308 for double) within the focused discipline
ExtremePositiveValueInIntegerFieldsFuzzer – iterate by every Integer discipline and ship requests with the very best worth potential (9223372036854775807 for int32 and 18446744073709551614 for int64) within the focused discipline
IntegerFieldsLeftBoundaryFuzzer – iterate by every Integer discipline and ship requests with exterior the vary values on the left aspect within the focused discipline
IntegerFieldsRightBoundaryFuzzer – iterate by every Integer discipline and ship requests with exterior the vary values on the suitable aspect within the focused discipline
InvalidValuesInEnumsFieldsFuzzer – iterate by every ENUM discipline and ship invalid values
LeadingWhitespacesInFieldsTrimValidateFuzzer – iterate by every discipline and ship requests with Unicode whitespaces and invisible separators prefixing the present worth within the focused discipline
LeadingControlCharsInFieldsTrimValidateFuzzer – iterate by every discipline and ship requests with Unicode management chars prefixing the present worth within the focused discipline
LeadingSingleCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values prefixed with single code factors emojis
LeadingMultiCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values prefixed with multi code factors emojis
MaxLengthExactValuesInStringFieldsFuzzer – iterate by every String fields which have maxLength declared and ship requests with values matching the maxLength measurement/worth within the focused discipline
MaximumExactValuesInNumericFieldsFuzzer – iterate by every Quantity and Integer fields which have most declared and ship requests with values matching the utmost measurement/worth within the focused discipline
MinLengthExactValuesInStringFieldsFuzzer – iterate by every String fields which have minLength declared and ship requests with values matching the minLength measurement/worth within the focused discipline
MinimumExactValuesInNumericFieldsFuzzer – iterate by every Quantity and Integer fields which have minimal declared and ship requests with values matching the minimal measurement/worth within the focused discipline
NewFieldsFuzzer – ship a ‘blissful’ circulate request and add a brand new discipline contained in the request referred to as ‘catsFuzzyField’
NullValuesInFieldsFuzzer – iterate by every discipline and ship requests with null values within the focused discipline
OnlyControlCharsInFieldsTrimValidateFuzzer – iterate by every discipline and ship values with management chars solely
OnlyWhitespacesInFieldsTrimValidateFuzzer – iterate by every discipline and ship values with unicode separators solely
OnlySingleCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values with single code level emojis solely
OnlyMultiCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values with multi code level emojis solely
RemoveFieldsFuzzer – iterate by every request fields and take away sure fields in response to the provided ‘fieldsFuzzingStrategy’
StringFieldsLeftBoundaryFuzzer – iterate by every String discipline and ship requests with exterior the vary values on the left aspect within the focused discipline
StringFieldsRightBoundaryFuzzer – iterate by every String discipline and ship requests with exterior the vary values on the suitable aspect within the focused discipline
StringFormatAlmostValidValuesFuzzer – iterate by every String discipline and get its ‘format’ worth (i.e. electronic mail, ip, uuid, date, datetime, and so forth); ship requests with values that are virtually legitimate (i.e. [email protected] for electronic mail, 888.1.1. for ip, and so forth) within the focused discipline
StringFormatTotallyWrongValuesFuzzer – iterate by every String discipline and get its ‘format’ worth (i.e. electronic mail, ip, uuid, date, datetime, and so forth); ship requests with values that are completely flawed (i.e. abcd for electronic mail, 1244. for ip, and so forth) within the focused discipline
StringsInNumericFieldsFuzzer – iterate by every Integer (int, lengthy) and Quantity discipline (float, double) and ship requests having the fuzz string worth within the focused discipline
TrailingWhitespacesInFieldsTrimValidateFuzzer – iterate by every discipline and ship requests with trailing with Unicode whitespaces and invisible separators within the focused discipline
TrailingControlCharsInFieldsTrimValidateFuzzer – iterate by every discipline and ship requests with trailing with Unicode management chars within the focused discipline
TrailingSingleCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values trailed with single code level emojis
TrailingMultiCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values trailed with multi code level emojis
VeryLargeStringsFuzzer – iterate by every String discipline and ship requests with very giant values (40000 characters) within the focused discipline
WithinControlCharsInFieldsSanitizeValidateFuzzer – iterate by every discipline and ship values containing unicode management chars
WithinSingleCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values containing single code level emojis
WithinMultiCodePointEmojisInFieldsTrimValidateFuzzer – iterate by every discipline and ship values containing multi code level emojis
ZalgoTextInStringFieldsValidateSanitizeFuzzer – iterate by every discipline and ship values containing zalgo textual content

You possibly can run solely these Fuzzers by supplying the --checkFields argument.

Header Fuzzers

CATS has at the moment 28 registered Header Fuzzers:

AbugidasCharsInHeadersFuzzer – iterate by every header and ship requests with abugidas chars within the focused header
CheckSecurityHeadersFuzzer – test all responses for good practices round Safety associated headers like: [{name=Cache-Control, value=no-store}, {name=X-XSS-Protection, value=1; mode=block}, {name=X-Content-Type-Options, value=nosniff}, {name=X-Frame-Options, value=DENY}]
DummyAcceptHeadersFuzzer – ship a request with a dummy Settle for header and anticipate to get 406 code
DummyContentTypeHeadersFuzzer – ship a request with a dummy Content material-Sort header and anticipate to get 415 code
DuplicateHeaderFuzzer – ship a ‘blissful’ circulate request and duplicate an current header
EmptyStringValuesInHeadersFuzzer – iterate by every header and ship requests with empty String values within the focused header
ExtraHeaderFuzzer – ship a ‘blissful’ circulate request and add an additional discipline contained in the request referred to as ‘Cats-Fuzzy-Header’
LargeValuesInHeadersFuzzer – iterate by every header and ship requests with giant values within the focused header
LeadingControlCharsInHeadersFuzzer – iterate by every header and prefix values with management chars
LeadingWhitespacesInHeadersFuzzer – iterate by every header and prefix worth with unicode separators
LeadingSpacesInHeadersFuzzer – iterate by every header and ship requests with areas prefixing the worth within the focused header
RemoveHeadersFuzzer – iterate by every header and take away totally different mixtures of them
OnlyControlCharsInHeadersFuzzer – iterate by every header and exchange worth with management chars
OnlySpacesInHeadersFuzzer – iterate by every header and exchange worth with areas
OnlyWhitespacesInHeadersFuzzer – iterate by every header and exchange worth with unicode separators
TrailingSpacesInHeadersFuzzer – iterate by every header and ship requests with trailing areas within the focused header
TrailingControlCharsInHeadersFuzzer – iterate by every header and path values with management chars
TrailingWhitespacesInHeadersFuzzer – iterate by every header and path values with unicode separators
UnsupportedAcceptHeadersFuzzer – ship a request with an unsupported Settle for header and anticipate to get 406 code
UnsupportedContentTypesHeadersFuzzer – ship a request with an unsupported Content material-Sort header and anticipate to get 415 code
ZalgoTextInHeadersFuzzer – iterate by every header and ship requests with zalgo textual content within the focused header

You possibly can run solely these Fuzzers by supplying the --checkHeaders argument.

HTTP Fuzzers

CATS has at the moment 6 registered HTTP Fuzzers:

BypassAuthenticationFuzzer – test if an authentication header is provided; if sure attempt to make requests with out it
DummyRequestFuzzer – ship a dummy json request {‘cats’: ‘cats’}
HappyFuzzer – ship a request with all fields and headers populated
HttpMethodsFuzzer – iterate by every undocumented HTTP methodology and ship an empty request
MalformedJsonFuzzer – ship a malformed json request which has the String ‘bla’ on the finish
NonRestHttpMethodsFuzzer – iterate by a listing of HTTP methodology particular to the WebDav protocol that aren’t anticipated to be carried out by REST APIs

You possibly can run solely these Fuzzers by supplying the --checkHttp argument.

ContractInfo Fuzzers or OpenAPI Linters

Often OpenAPI contract should observe a number of good practices in an effort to make it simple digestible by the service shoppers and act as a lot as potential as self-sufficient documentation:

observe good practices round naming the contract parts like paths, requests, responses
at all times use plural for the trail names, separate paths phrases by hyphens/underscores, use camelCase or snake_case for any json sorts and properties
present tags for all operations in an effort to keep away from breaking code technology on some languages and have a logical grouping of the API operations
present good description for all paths, strategies and request/response parts
present significant responses for POST, PATCH and PUT requests
present examples for all requests/response parts
present structural constraints for (ideally) all request/response properties (min, max, regex)
heaver some kind of CorrelationIds/TraceIds inside headers
have at the least a safety schema in place
keep away from having the API model a part of the paths
doc response codes for each “blissful” and “sad” flows
keep away from utilizing xml payload except there’s a actually good purpose (like documenting an previous API for instance)
json sorts and properties don’t use the identical naming (like having a Pet with a property named pet)

CATS has at the moment 9 registered ContractInfo Fuzzers:

HttpStatusCodeInValidRangeFuzzer – verifies that every one HTTP response codes are throughout the vary of 100 to 599
NamingsContractInfoFuzzer – verifies that every one OpenAPI contract parts observe REST API naming good practices
PathTagsContractInfoFuzzer – verifies that every one OpenAPI paths comprise tags parts and checks if the tags parts match those declared on the high stage
RecommendedHeadersContractInfoFuzzer – verifies that every one OpenAPI contract paths comprise really helpful headers like: CorrelationId/TraceId, and so forth.
RecommendedHttpCodesContractInfoFuzzer – verifies that the present path incorporates all really helpful HTTP response codes for all operations
SecuritySchemesContractInfoFuzzer – verifies if the OpenApi contract incorporates legitimate safety schemas for all paths, both globally configured or per path
TopLevelElementsContractInfoFuzzer – verifies that every one OpenAPI contract stage parts are current and supply significant info: API description, documentation, title, model, and so forth.
VersionsContractInfoFuzzer – verifies {that a} given path would not comprise versioning info
XmlContentTypeContractInfoFuzzer – verifies that every one OpenAPI contract paths responses and requests doesn’t provide software/xml as a Content material-Sort

You possibly can run solely these Fuzzers utilizing > cats lint --contract=CONTRACT.

Particular Fuzzers

FunctionalFuzzer

Writing Customized Checks

You possibly can leverage CATS super-powers of self-healing and payload technology in an effort to write useful checks.
That is achieved utilizing the so referred to as FunctionaFuzzer, which isn’t a Fuzzer per se, however was named as such for consistency.
The useful checks are written in a YAML file utilizing a easy DSL.
The DSL helps including identifiers, descriptions, assertions in addition to passing variables between checks.
The cool factor is that, by leveraging the truth that CATS generates legitimate payload, you solely have to override values for particular fields.
The remainder of the knowledge will likely be populated by CATS utilizing legitimate knowledge, identical to a ‘blissful’ circulate request.

It is vital to notice that reference knowledge will not get changed when utilizing the FunctionalFuzzer. So if there are reference knowledge fields, it’s essential to additionally provide these within the FunctionalFuzzer.

The FunctionalFuzzer will solely set off if a sound functionalFuzzer.yml file is provided. The file has the next syntax:

/path:
testNumber:
description: Brief description of the take a look at
prop: worth
prop#subprop: worth
prop7:
- value1
- value2
- value3
oneOfSelection:
component#sort: "Worth"
expectedResponseCode: HTTP_CODE
httpMethod: HTTP_NETHOD

And a typical run will appear to be:

> cats run functionalFuzzer.yml -c contract.yml -s http://localhost:8080

This can be a description of the weather throughout the functionalFuzzer.yml file:

you’ll be able to provide a description of the take a look at. This will likely be set because the Situation description. In the event you do not provide a description the testNumber will likely be used as a substitute.
you’ll be able to have a number of checks underneath the identical path: test1, test2, and so forth.
expectedResponseCode is necessary, in any other case the Fuzzer will ignore this take a look at. The expectedResponseCode tells CATS what to anticipate from the service when sending this take a look at.
at most one of many properties can have a number of values. When this example occurs, that take a look at will really develop into a listing of checks one for every of the values provided. For instance within the above instance prop7 has 3 values. It will really end in 3 checks, one for every worth.
take a look at throughout the file are executed within the declared order. This is the reason you’ll be able to have outputs from one take a look at act as inputs for the following one(s) (see the following part for particulars).
if the provided httpMethod would not exist within the OpenAPI given path, a warning will likely be issued and no take a look at will likely be executed
if the provided httpMethod will not be a sound HTTP methodology, a warning will likely be issued and no take a look at will likely be executed
if the request payload makes use of a oneOf component to permit a number of request sorts, you’ll be able to management which of the potential sorts the FunctionalFuzzer will apply to utilizing the oneOfSelection key phrase. The worth of the oneOfSelection key phrase should match the absolutely certified identify of the discriminator.
if no oneOfSelection is provided, and the request payload accepts a number of oneOf parts, than a customized take a look at will likely be created for every sort of payload
the file makes use of Json path syntax for all of the properties you’ll be able to provide; you’ll be able to separate parts by # as within the instance above as a substitute of .

Coping with oneOf, anyOf

When you will have request payloads which might take a number of object sorts, you need to use the oneOfSelection key phrase to specify which of the potential object sorts is required by the FunctionalFuzzer.
In the event you do not present this component, all mixtures will likely be thought of. In the event you provide a price, this should be precisely the one used within the discriminator.

Correlating Checks

As CATs principally depends on generated knowledge with small assist from some reference knowledge, testing advanced enterprise situations with the pre-defined Fuzzers will not be potential. Suppose we have now an endpoint that creates knowledge (doing a POST), and we need to test its existence (through GET).
We want a method to get some identifier from the POST name and ship it to the GET name. That is now potential utilizing the FunctionalFuzzer.
The functionalFuzzerFile can have an output entry the place you’ll be able to state a variable identify, and its absolutely certified identify from the response in an effort to set its worth.
You possibly can then refer the variable utilizing ${variable_name} from one other take a look at in an effort to use its worth.

Right here is an instance:

/pet:
test_1:
description: Create a Pet
httpMethod: POST
identify: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id
/pet/{id}:
test_2:
description: Get a Pet
id: ${petId}
expectedResponseCode: 200

Suppose the test_1 execution outputs:

When executing test_1 the worth of the pet id will likely be saved within the petId variable (worth 2).
When executing test_2 the id parameter will likely be changed with the petId variable (worth 2) from the earlier case.

Please be aware: variables are seen throughout all customized checks; please watch out with the naming as they may get overridden.

Verifying responses

The FunctionalFuzzer can confirm extra than simply the expectedResponseCode. That is achieved utilizing the confirm component. That is an prolonged model of the above functionalFuzzer.yml file.

/pet:
test_1:
description: Create a Pet
httpMethod: POST
identify: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id
confirm:
pet#identify: "Child"
pet#id: "[0-9]+"
/pet/{id}:
test_2:
description: Get a Pet
id: ${petId}
expectedResponseCode: 200

Contemplating the above file:

the FunctionalFuzzer will test if the response has the two parts pet#identify and pet#id
if the weather are discovered, it is going to test that the pet#identify has the Child worth and that the pet#id is numeric

The next json response will go test_1:

{
"pet": 
{ 
"id" : 2,
"identify": "Child"
}
}

However this one will not (pet#identify is lacking):

You can too discuss with request fields within the confirm part through the use of the ${request#..} qualifier. Utilizing the above instance, by having the next confirm part:

/pet:
test_1:
description: Create a Pet
httpMethod: POST
identify: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id
confirm:
pet#identify: "${request#identify}"
pet#id: "[0-9]+"

It is going to confirm if the response incorporates a pet#identify component and that its worth equals My Pet as despatched within the request.

Some notes:

confirm parameters assist Java regexes as values
you’ll be able to provide a couple of parameter to test (as seen above)
if at the least one of many parameters will not be current within the response, CATs will report an error
if all parameters are discovered and have legitimate values, however the response code will not be matched, CATs will report a warning
if all of the parameters are discovered and match their values, and the response code is as anticipated, CATs will report successful

Working with additionalProperties in FunctionalFuzzer

You can too set additionalProperties fields by the functionalFuzzerFile utilizing the identical syntax as for Setting additionalProperties in Reference Information.

FunctionalFuzzer Reserved key phrases

The next key phrases are reserved in FunctionalFuzzer checks: output, expectedResponseCode, httpMethod, description, oneOfSelection, confirm, additionalProperties, topElement and mapValues.

Safety Fuzzer

Though CATs will not be a safety testing software, you need to use it to check fundamental safety situations by fuzzing particular fields with totally different units of nasty strings.
The behaviour is just like the FunctionalFuzzer. You should utilize the very same parts for output variables, take a look at correlation, confirm responses and so forth, with the addition that it’s essential to additionally specify a targetFields and/or targetFieldTypes and a stringsList component.
A typical securityFuzzerFile will appear to be this:

/pet:
test_1:
description: Run XSS situations
identify: "My Pet"
expectedResponseCode: 200
httpMethod: all
targetFields:
- pet#id
- pet#description
stringsFile: xss.txt

And a typical run:

> cats run securityFuzzerFile.yml -c contract.yml -s http://localhost:8080

You can too provide output, httpMethod, oneOfSelection and/or confirm (with the identical behaviour as throughout the FunctionalFuzzer) if they’re related to your case.

The file makes use of Json path syntax for all of the properties you’ll be able to provide; you’ll be able to separate parts by # as within the instance as a substitute of ..

That is what the SecurityFuzzer will do after parsing the above securityFuzzerFile:

it is going to add the fastened worth “My Pet” to all of the request for the sector identify
for every discipline specified within the targetFields i.e. pet#id and pet#description it is going to create requests for every line from the xss.txt file and provide these values in every discipline
when you think about the xss.txt pattern file included within the CATs repo, which means it is going to ship 21 requests focusing on pet#id and 21 requests focusing on pet#description i.e. a complete of 42 checks
for every of those 42 checks, the SecurityFuzzer will anticipate a 200 response code. If one other response code is returned, then CATs will report the take a look at as error.

In order for you the above logic to use to all paths, you need to use all as the trail identify:

all:
test_1:
description: Run XSS situations
identify: "My Pet"
expectedResponseCode: 200
httpMethod: all
targetFields:
- pet#id
- pet#description
stringsFile: xss.txt

As an alternative of specifying the sector names, you’ll be able to broader to scope to focus on sure fields sorts. For instance, if we need to take a look at for XSS in all string fields, you’ll be able to have the next securityFuzzerFile:

all:
test_1:
description: Run XSS situations
identify: "My Pet"
expectedResponseCode: 200
httpMethod: all
targetFieldTypes:
- string
stringsFile: xss.txt

As an concept on learn how to create safety checks, you’ll be able to break up the nasty strings into a number of information of curiosity in your explicit context.
You possibly can have a sql_injection.txt, a xss.txt, a command_injection.txt and so forth. For every of those information, you’ll be able to create a take a look at entry within the securityFuzzerFile the place you embody the fields you assume are significant for most of these checks.
(It was a deliberate selection (for now) to not embody all fields by default.) The expectedResponseCode ought to be tweaked in response to your explicit context.
Your service would possibly sanitize knowledge earlier than validation, so may be completely legitimate to anticipate a 200 or would possibly validate the fields instantly, so may be completely legitimate to anticipate a 400.
A 500 will often imply one thing was not dealt with correctly and would possibly sign a potential bug.

Working with additionalProperties in SecurityFuzzer

You can too set additionalProperties fields by the functionalFuzzerFile utilizing the identical syntax as for Setting additionalProperties in Reference Information.

SecurityFuzzer Reserved key phrases

The next key phrases are reserved in SecurityFuzzer checks: output, expectedResponseCode, httpMethod, description, confirm, oneOfSelection, targetFields, targetFieldTypes, stringsFile, additionalProperties, topElement and mapValues.

TemplateFuzzer

The TemplateFuzzer can be utilized to fuzz non-OpenAPI endpoints. If the goal API doesn’t have an OpenAPI spec accessible, you need to use a request template to run a restricted set of fuzzers.
The syntax for operating the TemplateFuzzer is as follows (similar to curl:

> cats fuzz -H header=worth -X POST -d '{"field1":"value1","field2":"value2","field3":"value3"}' -t "field1,field2,header" -i "2XX,4XX" http://service-url

The command will:

ship a POST request to http://service-url
use the {"field1":"value1","field2":"value2","field3":"value3"} as a template
exchange one after the other field1,field2,header with fuzz knowledge and ship every request to the service endpoint
ignore 2XX,4XX response codes and report an error when the acquired response code will not be on this listing

It was a deliberate option to restrict the fields for which the Fuzzer will run by supplying them utilizing the -t argument. For nested objects, provide absolutely certified names: discipline.subfield.

Headers will also be fuzzed utilizing the identical mechanism because the fields.

This Fuzzer will ship the next sort of knowledge:

null values
empty values
zalgo textual content
abugidas characters
giant random unicode knowledge
very giant strings (80k characters)
single and multi code level emojis
unicode management characters
unicode separators
unicode whitespaces

For a full listing of choices run > cats fuzz -h.

You can too provide your individual dictionary of knowledge utilizing the -w file argument.

HTTP strategies with our bodies will solely be fuzzed on the request payload and headers stage.

HTTP strategies with out our bodies will likely be fuzzed at path and question parameters and headers stage. On this case you needn’t provide a -d argument.

That is an instance for a GET request:

> cats fuzz -X GET -t "path1,query1" -i "2XX,4XX" http://service-url/paths1?query1=take a look at&query2

There are sometimes circumstances the place some fields have to comprise related enterprise values to ensure that a request to succeed. You possibly can present such values utilizing a reference knowledge file specified by the --refData argument. The reference knowledge file is a YAML-format file that incorporates particular fastened values for various paths within the request doc. The file construction is as follows:

/path/0.1/auth:
prop#subprop: 12
prop2: 33
prop3#subprop1#subprop2: "take a look at"
/path/0.1/cancel:
prop#take a look at: 1

For every path you’ll be able to provide customized values for properties and sub-properties which may have precedence over values provided by another Fuzzer.
Contemplate this request payload:

{
"deal with": {
"cellphone": "123",
"postCode": "408",
"avenue": "cool avenue"    
},
"identify": "Joe"
}

and the next reference knowledge file file:

/path/0.1/auth:
deal with#avenue: "My Avenue"
identify: "John"

It will end in any fuzzed request to the /path/0.1/auth endpoint being up to date to comprise the provided fastened values:

{
"deal with": {
"cellphone": "123",
"postCode": "408",
"avenue": "My Avenue"    
},
"identify": "John"
}

The file makes use of Json path syntax for all of the properties you’ll be able to provide; you’ll be able to separate parts by # as within the instance above as a substitute of ..

You should utilize setting (system) variables in a ref knowledge file utilizing: $$VARIABLE_NAME. (discover double $$)

Setting additionalProperties

As further properties are maps i.e. they do not even have a construction, CATS can not at the moment generate legitimate values. If the weather inside such an information construction are important for a request,
you’ll be able to provide them through the refData file utilizing the next syntax:

/path/0.1/auth:
deal with#avenue: "My Avenue"
identify: "John"
additionalProperties:
topElement: metadata
mapValues:
take a look at: "value1"
anotherTest: "value2"

The additionalProperties component should comprise the precise key-value pairs to be despatched throughout the requests and likewise a high component if wanted. topElement will not be necessary.
The above instance will output the next json (contemplating additionally the above examples):

{
"deal with": {
"cellphone": "123",
"postCode": "408",
"avenue": "My Avenue"    
},
"identify": "John",
"metadata": {
"take a look at": "value1",
"anotherTest": "value2"
}   
}

RefData reserved key phrases

The next key phrases are reserved in a reference knowledge file: additionalProperties, topElement and mapValues.

Sending ref knowledge for ALL paths

You can too have the flexibility to ship the identical reference knowledge for ALL paths (identical to you do with the headers). You possibly can obtain this through the use of all as a key within the refData file:

It will attempt to exchange deal with#zip in all requests (if the sector is current).

Eradicating fields

There are (uncommon) circumstances when some fields could not make sense collectively. One thing like: when you ship firstName and lastName, you aren’t allowed to additionally ship identify.
As OpenAPI doesn’t have the aptitude to ship request fields that are depending on one another, you need to use the refData file to instruct CATS to take away fields earlier than sending a request to the service.
You possibly can obtain this through the use of the cats_remove_field as a price for the fields you need to take away. For the above case the refData discipline will look as follows:

all:
identify: "cats_remove_field"

Making a Ref Information file with the FunctionalFuzzer

You possibly can leverage the truth that the FunctionalFuzzer can run useful flows in an effort to create dynamic --refData information which will not want handbook setting the reference knowledge values.
The --refData file should be created with variables ${variable} as a substitute of fastened values and people variables should be output variables within the functionalFuzzer.yml file.
To ensure that the FunctionalFuzzer to correctly exchange the variables names with their values it’s essential to provide the --refData file as an argument when the FunctionalFuzzer runs.

> cats run functionalFuzzer.yml -c contract.yml -s http://localhost:8080 --refData=refData.yml

The functionalFuzzer.yml file:

/pet:
test_1:
description: Create a Pet
httpMethod: POST
identify: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id

The refData.yml file:

After operating CATS utilizing the command and the two information above, you’ll get a refData_replace.yml file the place the id will get the worth returned into the petId variable.

The refData_replaced.yml:

Now you can use the refData_replaced.yml as a --refData file for operating CATS with the remainder of the Fuzzers.

This can be utilized to ship customized fastened headers with every payload. It’s helpful when you will have authentication tokens you need to use to authenticate the API calls. You should utilize path particular headers or frequent headers that will likely be added to every name utilizing an all component. Particular paths will take priority over the all component.
Pattern headers file:

all:
Settle for: software/json
/path/0.1/auth:
jwt: XXXXXXXXXXXXX
/path/0.2/cancel:
jwt: YYYYYYYYYYYYY

It will add the Settle for header to all calls and the jwt header to the desired paths. You should utilize setting (system) variables in a headers file utilizing: $$VARIABLE_NAME. (discover double $$)

DELETE is the one HTTP verb that’s supposed to take away assets and executing the identical DELETE request twice will end in the second to fail because the useful resource is now not accessible.
Will probably be fairly heavy to produce a big listing of identifiers throughout the --refData file and because of this the advice was to skip the DELETE methodology when operating CATS.

However beginning with model 7.0.2 CATS has some intelligence in coping with DELETE. As a way to have sufficient legitimate entities CATS will save the corresponding POST requests in an inner Queue, and
everytime a DELETE request it will likely be executed it is going to ballot knowledge from there. As a way to have this really working, your contract should adjust to frequent sense conventions:

the DELETE path is definitely the POST path plus an identifier: if POST is /pets, then DELETE is predicted to be /pets/{petId}.
CATS will attempt to match the {petId} parameter throughout the physique returned by the POST request whereas doing numerous mixtures of the petId identify. It is going to attempt to seek for the next entries: petId, id, pet-id, pet_id with totally different circumstances.
If any of these entries is discovered inside a saved POST consequence, it is going to exchange the {petId} with that worth

For instance, suppose {that a} POST to /pets responds with:

{
"pet_id": 2,
"identify": "Chuck"
}

When doing a DELETE request, CATS will uncover that {petId} and pet_id are used as identifiers for the Pet useful resource, and can do the DELETE at /pets/2.

If these conventions are adopted (which additionally align to good REST naming practices), it’s anticipated that DELETE and POSTrequests will likely be on-par for many of the entities.

Some APIs would possibly use content material negotiation versioning which suggests codecs like software/v11+json within the Settle for header.

You possibly can deal with this in CATS as follows:

if the OpenAPI contract defines its content material as:

 requestBody:
required: true
content material:
software/v5+json:
schema:
$ref: '#/parts/RequestV5'
software/v6+json:
schema:
$ref: '#/parts/RequestV6'

by having clear separation between variations, you’ll be able to go the --contentType argument with the model you need to take a look at: cats ... --contentType="software/v6+json".

If the OpenAPI contract will not be model conscious (you already exported it particular to a model) and the content material appears to be like as:

 requestBody:
required: true
content material:
software/json:
schema:
$ref: '#/parts/RequestV5'

and you continue to have to go the software/v5+json Settle for header, you need to use the --headers file so as to add it:

all:
Settle for: "software/v5+json"

There is not a consensus on how you need to deal with conditions if you path or prefix legitimate values with areas.
One technique will likely be to have the service trimming areas earlier than doing the validation, whereas another companies will simply validate them as they’re.
You possibly can management how CATS ought to anticipate such circumstances to be dealt with by the service utilizing the --edgeSpacesStrategy argument.
You possibly can set this to trimAndValidate or validateAndTrim relying on the way you anticipate the service to behave:

trimAndValidate signifies that the service will first trim the areas and after that run the validation
validateAndTrim signifies that the service runs the validation first with none trimming of areas

This can be a world setting i.e. configured when CATS begins and all Fuzzer expects a constant behaviour from all of the service endpoints.

There are circumstances when sure elements of the request URL are parameterized. For instance a case like: /{model}/pets. {model} is meant to have the identical worth for all requests.
This is the reason you’ll be able to provide precise values to interchange such parameters utilizing the --urlParams argument.
You possibly can provide a ; separated listing of identify:worth pairs to interchange the identify parameters with their corresponding worth.
For instance supplying --urlParams=model:v1.0 will exchange the model parameter from the above instance with the worth v1.0.

CATS additionally helps schemas with oneOf, allOf and anyOf composition. CATS wil think about all potential mixtures when creating the fuzzed payloads.

The next configuration information: securityFuzzerFile, functionalFuzzerFile, refData assist setting dynamic values for the interior fields.
For now the assist solely exists for java.time.* and org.apache.commons.lang3.*, however extra kinds of parts will come within the close to future.

Let’s suppose you will have a date/date-time discipline, and also you need to set it to 10 days from now. You are able to do this by setting this as a price T(java.time.OffsetDateTime).now().plusDays(10).
It will return an ISO compliant time in UTC format.

A functionalFuzzer utilizing this could appear to be:

/path:
testNumber:
description: Brief description of the take a look at
prop: worth
prop#subprop: "T(java.time.OffsetDateTime).now().plusDays(10)"
prop7:
- value1
- value2
- value3
oneOfSelection:
component#sort: "Worth"
expectedResponseCode: HTTP_CODE
httpMethod: HTTP_NETHOD

You can too test the responses utilizing an identical syntax and likewise accounting for the precise values returned within the response. This can be a syntax than can take a look at if a returned date is after the present date:
T(java.time.LocalDate).now().isBefore(T(java.time.LocalDate).parse(expiry.toString())). It is going to test if the expiry discipline returned within the json response, parsed as date, is after the present date.

The syntax of dynamically setting dates is compliant with the Spring Expression Language specs.

If you want to run CATS behind a proxy, you’ll be able to provide the next arguments: --proxyHost and --proxyPort.
A typical run with proxy settings on localhost:8080 will look as follows:

> cats --contract=YAML_FILE --server=SERVER_URL --proxyHost=localhost --proxyPort=8080

HTTP header(s) based mostly authentication

CATS helps any type of HTTP header(s) based mostly authentication (fundamental auth, oauth, customized JWT, apiKey, and so forth) utilizing the headers mechanism. You possibly can provide the particular HTTP header identify and worth
and apply to all endpoints.
Moreover, fundamental auth can be supported utilizing the --basicauth=USR:PWD argument.

One-Approach or Two-Approach SSL

By default, CATS trusts all server certificates and would not carry out hostname verification.

For 2-way SSL you’ll be able to specify a JKS file (Java Keystore) that holds the consumer’s non-public key utilizing the next arguments:

--sslKeystore Location of the JKS keystore holding certificates used when authenticating calls utilizing one-way or two-way SSL
--sslKeystorePwd The password of the sslKeystore
--sslKeyPwd The password of the non-public key throughout the sslKeystore

For particulars on learn how to load the certificates and personal key right into a Java Keystore you need to use this information: https://mrkandreev.identify/weblog/java-two-way-ssl/.

Native Binaries

When utilizing the native binaries (not the uberjar) there may be points when utilizing dynamic values within the CATS information.
This is because of the truth that GraalVM solely bundles no matter can uncover at compile time.
The next courses are at the moment supported:

java.util.Base64.Encoder.class, java.util.Base64.Decoder.class, java.util.Base64.class, org.apache.commons.lang3.RandomUtils.class, org.apache.commons.lang3.RandomStringUtils.class, 
org.apache.commons.lang3.DateFormatUtils.class, org.apache.commons.lang3.DateUtils.class, 
org.apache.commons.lang3.DurationUtils.class, java.time.LocalDate.class, java.time.LocalDateTime.class, java.time.OffsetDateTime.class

API specs

At this second, CATS solely works with OpenAPI specs and has restricted performance utilizing template payloads by the cats fuzz ... subcommand.

Media sorts and HTTP strategies

The Fuzzers has the next assist for media sorts and HTTP strategies:

software/json and software/x-www-form-urlencoded media sorts solely
HTTP strategies: POST, PUT, PATCH, GET and DELETE

Extra Parameters

If a response incorporates a free Map specified utilizing the additionalParameters tag CATS will challenge a WARN stage log message because it will not be capable of validate that the response matches the schema.

Regexes inside ‘sample’

CATS makes use of RgxGen in an effort to generate Strings based mostly on regexes. This has sure limitations principally with advanced patterns.

All customized information that can be utilized by CATS (functionalFuzzerFile, headers, refData, and so forth) are in a YAML format. When setting or getting values to/from JSON for enter and/or output variables, it’s essential to use a JsonPath syntax utilizing both # or . as separators.
You could find some selector examples right here: JsonPath.

Please discuss with CONTRIBUTING.md.