91 lines
No EOL
2.3 KiB
Markdown
91 lines
No EOL
2.3 KiB
Markdown
# Standard Tests Development Guide
|
||
|
||
Standard test cases are found in [aristaproto/tests/inputs](inputs), where each subdirectory represents a testcase, that is verified in isolation.
|
||
|
||
```
|
||
inputs/
|
||
bool/
|
||
double/
|
||
int32/
|
||
...
|
||
```
|
||
|
||
## Test case directory structure
|
||
|
||
Each testcase has a `<name>.proto` file with a message called `Test`, and optionally a matching `.json` file and a custom test called `test_*.py`.
|
||
|
||
```bash
|
||
bool/
|
||
bool.proto
|
||
bool.json # optional
|
||
test_bool.py # optional
|
||
```
|
||
|
||
### proto
|
||
|
||
`<name>.proto` — *The protobuf message to test*
|
||
|
||
```protobuf
|
||
syntax = "proto3";
|
||
|
||
message Test {
|
||
bool value = 1;
|
||
}
|
||
```
|
||
|
||
You can add multiple `.proto` files to the test case, as long as one file matches the directory name.
|
||
|
||
### json
|
||
|
||
`<name>.json` — *Test-data to validate the message with*
|
||
|
||
```json
|
||
{
|
||
"value": true
|
||
}
|
||
```
|
||
|
||
### pytest
|
||
|
||
`test_<name>.py` — *Custom test to validate specific aspects of the generated class*
|
||
|
||
```python
|
||
from tests.output_aristaproto.bool.bool import Test
|
||
|
||
def test_value():
|
||
message = Test()
|
||
assert not message.value, "Boolean is False by default"
|
||
```
|
||
|
||
## Standard tests
|
||
|
||
The following tests are automatically executed for all cases:
|
||
|
||
- [x] Can the generated python code be imported?
|
||
- [x] Can the generated message class be instantiated?
|
||
- [x] Is the generated code compatible with the Google's `grpc_tools.protoc` implementation?
|
||
- _when `.json` is present_
|
||
|
||
## Running the tests
|
||
|
||
- `pipenv run generate`
|
||
This generates:
|
||
- `aristaproto/tests/output_aristaproto` — *the plugin generated python classes*
|
||
- `aristaproto/tests/output_reference` — *reference implementation classes*
|
||
- `pipenv run test`
|
||
|
||
## Intentionally Failing tests
|
||
|
||
The standard test suite includes tests that fail by intention. These tests document known bugs and missing features that are intended to be corrected in the future.
|
||
|
||
When running `pytest`, they show up as `x` or `X` in the test results.
|
||
|
||
```
|
||
aristaproto/tests/test_inputs.py ..x...x..x...x.X........xx........x.....x.......x.xx....x...................... [ 84%]
|
||
```
|
||
|
||
- `.` — PASSED
|
||
- `x` — XFAIL: expected failure
|
||
- `X` — XPASS: expected failure, but still passed
|
||
|
||
Test cases marked for expected failure are declared in [inputs/config.py](inputs/config.py) |