Adding upstream version 1.1.0.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
f13b7abbd8
commit
77504588ab
GPG key ID:
FBB4F0E80A80222F
196 changed files with 10121 additions and 3780 deletions
|
|
@ -4,162 +4,250 @@
|
|||
~ that can be found in the LICENSE file.
|
||||
-->
|
||||
|
||||
# Tag management
|
||||
ANTA commands can be used with a `--tags` option. This option **filters the inventory** with the specified tag(s) when running the command.
|
||||
|
||||
## Overview
|
||||
Tags can also be used to **restrict a specific test** to a set of devices when using `anta nrfu`.
|
||||
|
||||
Some of the ANTA commands like `anta nrfu` command come with a `--tags` option.
|
||||
## Defining tags
|
||||
|
||||
For `nrfu`, this allows users to specify a set of tests, marked with a given tag, to be run on devices marked with the same tag. For instance, you can run tests dedicated to leaf devices on your leaf devices only and not on other devices.
|
||||
### Device tags
|
||||
|
||||
Tags are string defined by the user and can be anything considered as a string by Python. A [default one](#default-tags) is present for all tests and devices.
|
||||
Device tags can be defined in the inventory:
|
||||
|
||||
The next table provides a short summary of the scope of tags using CLI
|
||||
```yaml
|
||||
anta_inventory:
|
||||
hosts:
|
||||
- name: leaf1
|
||||
host: leaf1.anta.arista.com
|
||||
tags: ["leaf"]
|
||||
- name: leaf2
|
||||
host: leaf2.anta.arista.com
|
||||
tags: ["leaf"]
|
||||
- name: spine1
|
||||
host: spine1.anta.arista.com
|
||||
tags: ["spine"]
|
||||
```
|
||||
|
||||
Each device also has its own name automatically added as a tag:
|
||||
|
||||
```bash
|
||||
$ anta get inventory
|
||||
Current inventory content is:
|
||||
{
|
||||
'leaf1': AsyncEOSDevice(
|
||||
name='leaf1',
|
||||
tags={'leaf', 'leaf1'}, <--
|
||||
[...]
|
||||
host='leaf1.anta.arista.com',
|
||||
[...]
|
||||
),
|
||||
'leaf2': AsyncEOSDevice(
|
||||
name='leaf2',
|
||||
tags={'leaf', 'leaf2'}, <--
|
||||
[...]
|
||||
host='leaf2.anta.arista.com',
|
||||
[...]
|
||||
),
|
||||
'spine1': AsyncEOSDevice(
|
||||
name='spine1',
|
||||
tags={'spine1', 'spine'}, <--
|
||||
[...]
|
||||
host='spine1.anta.arista.com',
|
||||
[...]
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### Test tags
|
||||
|
||||
Tags can be defined in the test catalog to restrict tests to tagged devices:
|
||||
|
||||
```yaml
|
||||
anta.tests.system:
|
||||
- VerifyUptime:
|
||||
minimum: 10
|
||||
filters:
|
||||
tags: ['spine']
|
||||
- VerifyUptime:
|
||||
minimum: 9
|
||||
filters:
|
||||
tags: ['leaf']
|
||||
- VerifyReloadCause:
|
||||
filters:
|
||||
tags: ['spine', 'leaf']
|
||||
- VerifyCoredump:
|
||||
- VerifyAgentLogs:
|
||||
- VerifyCPUUtilization:
|
||||
- VerifyMemoryUtilization:
|
||||
- VerifyFileSystemUtilization:
|
||||
- VerifyNTP:
|
||||
|
||||
anta.tests.mlag:
|
||||
- VerifyMlagStatus:
|
||||
filters:
|
||||
tags: ['leaf']
|
||||
|
||||
anta.tests.interfaces:
|
||||
- VerifyL3MTU:
|
||||
mtu: 1500
|
||||
filters:
|
||||
tags: ['spine']
|
||||
```
|
||||
|
||||
> A tag used to filter a test can also be a device name
|
||||
|
||||
!!! tip "Use different input values for a specific test"
|
||||
Leverage tags to define different input values for a specific test. See the `VerifyUptime` example above.
|
||||
|
||||
## Using tags
|
||||
|
||||
| Command | Description |
|
||||
| ------- | ----------- |
|
||||
| `none` | Run all tests on all devices according `tag` definition in your inventory and test catalog. And tests with no tag are executed on all devices|
|
||||
| `--tags leaf` | Run all tests marked with `leaf` tag on all devices configured with `leaf` tag.<br/> All other tags are ignored |
|
||||
| `--tags leaf,spine` | Run all tests marked with `leaf` tag on all devices configured with `leaf` tag.<br/>Run all tests marked with `spine` tag on all devices configured with `spine` tag.<br/> All other tags are ignored |
|
||||
| No `--tags` option | Run all tests on all devices according to the `tag` definitions in your inventory and test catalog.<br/> Tests without tags are executed on all devices. |
|
||||
| `--tags leaf` | Run all tests marked with the `leaf` tag on all devices configured with the `leaf` tag.<br/> All other tests are ignored. |
|
||||
| `--tags leaf,spine` | Run all tests marked with the `leaf` tag on all devices configured with the `leaf` tag.<br/>Run all tests marked with the `spine` tag on all devices configured with the `spine` tag.<br/> All other tests are ignored. |
|
||||
|
||||
## Inventory and Catalog for tests
|
||||
### Examples
|
||||
|
||||
All commands in this page are based on the following inventory and test catalog.
|
||||
The following examples use the inventory and test catalog defined above.
|
||||
|
||||
=== "Inventory"
|
||||
#### No `--tags` option
|
||||
|
||||
```yaml
|
||||
---
|
||||
anta_inventory:
|
||||
hosts:
|
||||
- host: 192.168.0.10
|
||||
name: spine01
|
||||
tags: ['fabric', 'spine']
|
||||
- host: 192.168.0.11
|
||||
name: spine02
|
||||
tags: ['fabric', 'spine']
|
||||
- host: 192.168.0.12
|
||||
name: leaf01
|
||||
tags: ['fabric', 'leaf']
|
||||
- host: 192.168.0.13
|
||||
name: leaf02
|
||||
tags: ['fabric', 'leaf']
|
||||
- host: 192.168.0.14
|
||||
name: leaf03
|
||||
tags: ['fabric', 'leaf']
|
||||
- host: 192.168.0.15
|
||||
name: leaf04
|
||||
tags: ['fabric', 'leaf'
|
||||
```
|
||||
|
||||
=== "Test Catalog"
|
||||
|
||||
```yaml
|
||||
anta.tests.system:
|
||||
- VerifyUptime:
|
||||
minimum: 10
|
||||
filters:
|
||||
tags: ['fabric']
|
||||
- VerifyReloadCause:
|
||||
tags: ['leaf', spine']
|
||||
- VerifyCoredump:
|
||||
- VerifyAgentLogs:
|
||||
- VerifyCPUUtilization:
|
||||
filters:
|
||||
tags: ['spine', 'leaf']
|
||||
- VerifyMemoryUtilization:
|
||||
- VerifyFileSystemUtilization:
|
||||
- VerifyNTP:
|
||||
|
||||
anta.tests.mlag:
|
||||
- VerifyMlagStatus:
|
||||
|
||||
|
||||
anta.tests.interfaces:
|
||||
- VerifyL3MTU:
|
||||
mtu: 1500
|
||||
filters:
|
||||
tags: ['demo']
|
||||
```
|
||||
|
||||
## Default tags
|
||||
|
||||
By default, ANTA uses a default tag for both devices and tests. This default tag is `all` and it can be explicit if you want to make it visible in your inventory and also implicit since the framework injects this tag if it is not defined.
|
||||
|
||||
So this command will run all tests from your catalog on all devices. With a mapping for `tags` defined in your inventory and catalog. If no `tags` configured, then tests are executed against all devices.
|
||||
Tests without tags are run on all devices.
|
||||
Tests with tags will only run on devices with matching tags.
|
||||
|
||||
```bash
|
||||
$ anta nrfu -c .personal/catalog-class.yml table --group-by device
|
||||
|
||||
$ anta nrfu table --group-by device
|
||||
╭────────────────────── Settings ──────────────────────╮
|
||||
│ Running ANTA tests: │
|
||||
│ - ANTA Inventory contains 6 devices (AsyncEOSDevice) │
|
||||
│ - Tests catalog contains 10 tests │
|
||||
│ - ANTA Inventory contains 3 devices (AsyncEOSDevice) │
|
||||
│ - Tests catalog contains 11 tests │
|
||||
╰──────────────────────────────────────────────────────╯
|
||||
|
||||
┏━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
|
||||
┃ Device ┃ # of success ┃ # of skipped ┃ # of failure ┃ # of errors ┃ List of failed or error test cases ┃
|
||||
┡━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
|
||||
│ spine01 │ 5 │ 1 │ 1 │ 0 │ ['VerifyCPUUtilization'] │
|
||||
│ spine02 │ 5 │ 1 │ 1 │ 0 │ ['VerifyCPUUtilization'] │
|
||||
│ leaf01 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │
|
||||
│ leaf02 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │
|
||||
│ leaf03 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │
|
||||
│ leaf04 │ 6 │ 0 │ 1 │ 0 │ ['VerifyCPUUtilization'] │
|
||||
└─────────┴──────────────┴──────────────┴──────────────┴─────────────┴────────────────────────────────────┘
|
||||
--- ANTA NRFU Run Information ---
|
||||
Number of devices: 3 (3 established)
|
||||
Total number of selected tests: 27
|
||||
---------------------------------
|
||||
Summary per device
|
||||
┏━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
|
||||
┃ Device ┃ # of success ┃ # of skipped ┃ # of failure ┃ # of errors ┃ List of failed or error test cases ┃
|
||||
┡━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
|
||||
│ leaf1 │ 9 │ 0 │ 0 │ 0 │ │
|
||||
├────────┼──────────────┼──────────────┼──────────────┼─────────────┼────────────────────────────────────┤
|
||||
│ leaf2 │ 7 │ 1 │ 1 │ 0 │ VerifyAgentLogs │
|
||||
├────────┼──────────────┼──────────────┼──────────────┼─────────────┼────────────────────────────────────┤
|
||||
│ spine1 │ 9 │ 0 │ 0 │ 0 │ │
|
||||
└────────┴──────────────┴──────────────┴──────────────┴─────────────┴────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Use a single tag in CLI
|
||||
#### Single tag
|
||||
|
||||
The most used approach is to use a single tag in your CLI to filter tests & devices configured with this one.
|
||||
|
||||
In such scenario, ANTA will run tests marked with `$tag` only on devices marked with `$tag`. All other tests and devices will be ignored
|
||||
With a tag specified, only tests matching this tag will be run on matching devices.
|
||||
|
||||
```bash
|
||||
$ anta nrfu -c .personal/catalog-class.yml --tags leaf text
|
||||
$ anta nrfu --tags leaf text
|
||||
╭────────────────────── Settings ──────────────────────╮
|
||||
│ Running ANTA tests: │
|
||||
│ - ANTA Inventory contains 6 devices (AsyncEOSDevice) │
|
||||
│ - Tests catalog contains 10 tests │
|
||||
│ - ANTA Inventory contains 3 devices (AsyncEOSDevice) │
|
||||
│ - Tests catalog contains 11 tests │
|
||||
╰──────────────────────────────────────────────────────╯
|
||||
|
||||
leaf01 :: VerifyUptime :: SUCCESS
|
||||
leaf01 :: VerifyReloadCause :: SUCCESS
|
||||
leaf01 :: VerifyCPUUtilization :: SUCCESS
|
||||
leaf02 :: VerifyUptime :: SUCCESS
|
||||
leaf02 :: VerifyReloadCause :: SUCCESS
|
||||
leaf02 :: VerifyCPUUtilization :: SUCCESS
|
||||
leaf03 :: VerifyUptime :: SUCCESS
|
||||
leaf03 :: VerifyReloadCause :: SUCCESS
|
||||
leaf03 :: VerifyCPUUtilization :: SUCCESS
|
||||
leaf04 :: VerifyUptime :: SUCCESS
|
||||
leaf04 :: VerifyReloadCause :: SUCCESS
|
||||
leaf04 :: VerifyCPUUtilization :: SUCCESS
|
||||
--- ANTA NRFU Run Information ---
|
||||
Number of devices: 3 (2 established)
|
||||
Total number of selected tests: 6
|
||||
---------------------------------
|
||||
|
||||
leaf1 :: VerifyReloadCause :: SUCCESS
|
||||
leaf1 :: VerifyUptime :: SUCCESS
|
||||
leaf1 :: VerifyMlagStatus :: SUCCESS
|
||||
leaf2 :: VerifyReloadCause :: SUCCESS
|
||||
leaf2 :: VerifyUptime :: SUCCESS
|
||||
leaf2 :: VerifyMlagStatus :: SKIPPED (MLAG is disabled)
|
||||
```
|
||||
|
||||
In this case, only `leaf` devices defined in your [inventory](#inventory-and-catalog-for-tests) are used to run tests marked with `leaf` in your [test catalog](#inventory-and-catalog-for-tests)
|
||||
In this case, only `leaf` devices defined in the inventory are used to run tests marked with the `leaf` in the test catalog.
|
||||
|
||||
## Use multiple tags in CLI
|
||||
#### Multiple tags
|
||||
|
||||
A more advanced usage of the tag feature is to list multiple tags in your CLI using `--tags $tag1,$tag2` syntax.
|
||||
|
||||
In such scenario, all devices marked with `$tag1` will be selected and ANTA will run tests with `$tag1`, then devices with `$tag2` will be selected and will be tested with tests marked with `$tag2`
|
||||
It is possible to use multiple tags using the `--tags tag1,tag2` syntax.
|
||||
|
||||
```bash
|
||||
anta nrfu -c .personal/catalog-class.yml --tags leaf,fabric text
|
||||
$ anta nrfu --tags leaf,spine text
|
||||
╭────────────────────── Settings ──────────────────────╮
|
||||
│ - ANTA Inventory contains 3 devices (AsyncEOSDevice) │
|
||||
│ - Tests catalog contains 11 tests │
|
||||
╰──────────────────────────────────────────────────────╯
|
||||
|
||||
spine01 :: VerifyUptime :: SUCCESS
|
||||
spine02 :: VerifyUptime :: SUCCESS
|
||||
leaf01 :: VerifyUptime :: SUCCESS
|
||||
leaf01 :: VerifyReloadCause :: SUCCESS
|
||||
leaf01 :: VerifyCPUUtilization :: SUCCESS
|
||||
leaf02 :: VerifyUptime :: SUCCESS
|
||||
leaf02 :: VerifyReloadCause :: SUCCESS
|
||||
leaf02 :: VerifyCPUUtilization :: SUCCESS
|
||||
leaf03 :: VerifyUptime :: SUCCESS
|
||||
leaf03 :: VerifyReloadCause :: SUCCESS
|
||||
leaf03 :: VerifyCPUUtilization :: SUCCESS
|
||||
leaf04 :: VerifyUptime :: SUCCESS
|
||||
leaf04 :: VerifyReloadCause :: SUCCESS
|
||||
leaf04 :: VerifyCPUUtilization :: SUCCESS
|
||||
--- ANTA NRFU Run Information ---
|
||||
Number of devices: 3 (3 established)
|
||||
Total number of selected tests: 15
|
||||
---------------------------------
|
||||
|
||||
leaf1 :: VerifyReloadCause :: SUCCESS
|
||||
leaf1 :: VerifyMlagStatus :: SUCCESS
|
||||
leaf1 :: VerifyUptime :: SUCCESS
|
||||
leaf1 :: VerifyL3MTU :: SUCCESS
|
||||
leaf1 :: VerifyUptime :: SUCCESS
|
||||
leaf2 :: VerifyReloadCause :: SUCCESS
|
||||
leaf2 :: VerifyMlagStatus :: SKIPPED (MLAG is disabled)
|
||||
leaf2 :: VerifyUptime :: SUCCESS
|
||||
leaf2 :: VerifyL3MTU :: SUCCESS
|
||||
leaf2 :: VerifyUptime :: SUCCESS
|
||||
spine1 :: VerifyReloadCause :: SUCCESS
|
||||
spine1 :: VerifyMlagStatus :: SUCCESS
|
||||
spine1 :: VerifyUptime :: SUCCESS
|
||||
spine1 :: VerifyL3MTU :: SUCCESS
|
||||
spine1 :: VerifyUptime :: SUCCESS
|
||||
```
|
||||
|
||||
## Obtaining all configured tags
|
||||
|
||||
As most ANTA commands accommodate tag filtering, this command is useful for enumerating all tags configured in the inventory. Running the `anta get tags` command will return a list of all tags configured in the inventory.
|
||||
|
||||
### Command overview
|
||||
|
||||
```bash
|
||||
Usage: anta get tags [OPTIONS]
|
||||
|
||||
Get list of configured tags in user inventory.
|
||||
|
||||
Options:
|
||||
-u, --username TEXT Username to connect to EOS [env var: ANTA_USERNAME;
|
||||
required]
|
||||
-p, --password TEXT Password to connect to EOS that must be provided. It
|
||||
can be prompted using '--prompt' option. [env var:
|
||||
ANTA_PASSWORD]
|
||||
--enable-password TEXT Password to access EOS Privileged EXEC mode. It can
|
||||
be prompted using '--prompt' option. Requires '--
|
||||
enable' option. [env var: ANTA_ENABLE_PASSWORD]
|
||||
--enable Some commands may require EOS Privileged EXEC mode.
|
||||
This option tries to access this mode before sending
|
||||
a command to the device. [env var: ANTA_ENABLE]
|
||||
-P, --prompt Prompt for passwords if they are not provided. [env
|
||||
var: ANTA_PROMPT]
|
||||
--timeout FLOAT Global API timeout. This value will be used for all
|
||||
devices. [env var: ANTA_TIMEOUT; default: 30.0]
|
||||
--insecure Disable SSH Host Key validation. [env var:
|
||||
ANTA_INSECURE]
|
||||
--disable-cache Disable cache globally. [env var:
|
||||
ANTA_DISABLE_CACHE]
|
||||
-i, --inventory FILE Path to the inventory YAML file. [env var:
|
||||
ANTA_INVENTORY; required]
|
||||
--tags TEXT List of tags using comma as separator:
|
||||
tag1,tag2,tag3. [env var: ANTA_TAGS]
|
||||
--help Show this message and exit.
|
||||
```
|
||||
|
||||
### Example
|
||||
|
||||
To get the list of all configured tags in the inventory, run the following command:
|
||||
|
||||
```bash
|
||||
$ anta get tags
|
||||
Tags found:
|
||||
[
|
||||
"leaf",
|
||||
"leaf1",
|
||||
"leaf2",
|
||||
"spine",
|
||||
"spine1"
|
||||
]
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue