Add minunit test framework (#168)
* add minunit tests * fix logging * ignore unexisting time service on embedded targets * fix warning, issue with printf * add exitcode * migrate to printf * indicate test by leds * add testing description * redesigned minunit tests to allow testing in separate files * add test step for local target * add failure test * add restore test_check * testing description Co-authored-by: rusdacent <rusdacentx0x08@gmail.com> Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com>
This commit is contained in:
coreglitch
GitHub
126
wiki/Testing.md
126
wiki/Testing.md
@@ -1,121 +1,11 @@
|
||||
# Bootloader testcase
|
||||
# Integration testing
|
||||
|
||||
1. `# Clean flash`
|
||||
2. `make -C bootloader flash` `# Load bootloader`
|
||||
3. `# reboot device`
|
||||
* Press right
|
||||
* Press left
|
||||
* Wait 0.1 s
|
||||
* Release left
|
||||
* Release right
|
||||
4. Wait 0.5 s
|
||||
5. `# Expect no FW`
|
||||
* Expect: no uart welcome message
|
||||
* Expect: red led on
|
||||
* Expect: no USB
|
||||
6. `# reboot device and go to DFU`
|
||||
* Press left
|
||||
* Press right
|
||||
* Wait 0.1 s
|
||||
* Release left
|
||||
* Wait 0.5 s
|
||||
* Release right
|
||||
7. Wait 0.5 s
|
||||
8. `# Expect DFU`
|
||||
* Expect: blue led on
|
||||
* Expect: USB: DFU
|
||||
9. `target_f2/deploy-dfu.sh` `# load FW`
|
||||
10. `# reboot device`
|
||||
* Press right
|
||||
* Press left
|
||||
* Wait 0.1 s
|
||||
* Release left
|
||||
* Release right
|
||||
11. Wait 0.5 s
|
||||
12. `# Expect FW`
|
||||
* Expect: uart welcome message
|
||||
* Expect: USB Flipper CDC
|
||||
* **[Bootloader testcase](Bootloader-test)**
|
||||
* **[Input testcase](Input-test)**
|
||||
* **[General testcase](General-building-test)**
|
||||
|
||||
# Input testcase
|
||||
# Unit testing
|
||||
|
||||
1. `docker-compose exec dev make -C target_f2 example_input_dump`
|
||||
2. Flash
|
||||
3. For x in ```
|
||||
[
|
||||
(Up, "00"),
|
||||
(Down, "01"),
|
||||
(Right, "02"),
|
||||
(Left, "03"),
|
||||
(Ok, "04"),
|
||||
(Back, "05"),
|
||||
]
|
||||
```
|
||||
* Press ${x[0]}
|
||||
* wait 0.05
|
||||
* Expect: Uart: "event: ${x[1]} pressed"
|
||||
* wait 0.05
|
||||
* Release ${x[0]}
|
||||
* wait 0.05
|
||||
* Expect: Uart: "event: ${x[1]} released"
|
||||
* wait 0.05
|
||||
|
||||
TODO: add debouncing check (multiple press and check there is no multiple events)
|
||||
|
||||
# General building testcase
|
||||
|
||||
Local target:
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_TEST=1 clean`
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_TEST=1 run`
|
||||
* check tests pass/fail
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_BLINK=1 run`
|
||||
* GPIO on and off
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_UART_WRITE=1 run`
|
||||
* GPIO on/off and `counter: %` writes
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_IPC=1 run`
|
||||
* ASCII display draw
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_INPUT_DUMP=1 run` not implemented
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_QRCODE=1 run`
|
||||
* Some writes to display
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_DISPLAY=1 run`
|
||||
* Some writes to display
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_FATFS=1 flash`
|
||||
* TODO: FatFs emulation and test not implemented
|
||||
|
||||
F2 target:
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_TEST=1 clean`
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_TEST=1 flash`
|
||||
* check UART for test pass/fail
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_BLINK=1 flash`
|
||||
* Red LED blink (1s period)
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_UART_WRITE=1 flash`
|
||||
* Red LED shortly blinking, `counter: %` writes to UART
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_IPC=1 flash`
|
||||
* ASCII display draw in UART
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_INPUT_DUMP=1 flash`
|
||||
* Press all buttons, `state` and `event` writes to UART
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_QRCODE=1 flash`
|
||||
* QR code show on the screen
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_DISPLAY=1 flash`
|
||||
* `Hello world` show on the screen
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_FATFS=1 flash`
|
||||
* `Init sd card error` on the screen
|
||||
* Insert SD-card
|
||||
* Reboot
|
||||
* Show file list on the screen
|
||||
* Scroll by pressing up and down
|
||||
1. We use [minunit]() as testing framework
|
||||
2. Tests root placed in `applications/tests/minuint_test.c`
|
||||
3. There is `Run local tests` job in `CI` pipeline (`.github/workflows/ci.yml`)
|
||||
|
||||
@@ -35,7 +35,7 @@ For more HW- and RTOS- specific checks we run real FW in [Renode](https://interr
|
||||
|
||||
Eventually we run real FW on remote debug/test bench (#26): flipper board + RPi + some stuff to control and check real hardware.
|
||||
|
||||
# Debug/test bench (not implemented)
|
||||
# Debug/test bench (in progress)
|
||||
|
||||
* 24×7 connected target Flipper device and accessible via Internet. Raspberry PI or some Linux single-board PC can be used as basic high-level control board.
|
||||
* Tool can push/click each user buttons by hardware by "control board" (low level). Usage of optocouples/reed-switch relays is fine for that.
|
||||
@@ -61,3 +61,7 @@ Eventually we run real FW on remote debug/test bench (#26): flipper board + RPi
|
||||
2. Run CI tests:
|
||||
* For test automation we can use RobotDemo or simple expect tool/python scripts/etc.
|
||||
* Apply test cases and submit its results.
|
||||
|
||||
# Testing
|
||||
|
||||
You can read about testing in [Testing](Testing) page.
|
||||
|
||||
35
wiki/testing/Bootloader-test.md
Normal file
35
wiki/testing/Bootloader-test.md
Normal file
@@ -0,0 +1,35 @@
|
||||
1. `# Clean flash`
|
||||
2. `make -C bootloader flash` `# Load bootloader`
|
||||
3. `# reboot device`
|
||||
* Press right
|
||||
* Press left
|
||||
* Wait 0.1 s
|
||||
* Release left
|
||||
* Release right
|
||||
4. Wait 0.5 s
|
||||
5. `# Expect no FW`
|
||||
* Expect: no uart welcome message
|
||||
* Expect: red led on
|
||||
* Expect: no USB
|
||||
6. `# reboot device and go to DFU`
|
||||
* Press left
|
||||
* Press right
|
||||
* Wait 0.1 s
|
||||
* Release left
|
||||
* Wait 0.5 s
|
||||
* Release right
|
||||
7. Wait 0.5 s
|
||||
8. `# Expect DFU`
|
||||
* Expect: blue led on
|
||||
* Expect: USB: DFU
|
||||
9. `target_f2/deploy-dfu.sh` `# load FW`
|
||||
10. `# reboot device`
|
||||
* Press right
|
||||
* Press left
|
||||
* Wait 0.1 s
|
||||
* Release left
|
||||
* Release right
|
||||
11. Wait 0.5 s
|
||||
12. `# Expect FW`
|
||||
* Expect: uart welcome message
|
||||
* Expect: USB Flipper CDC
|
||||
59
wiki/testing/General-building-test.md
Normal file
59
wiki/testing/General-building-test.md
Normal file
@@ -0,0 +1,59 @@
|
||||
Local target:
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_TEST=1 clean`
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_TEST=1 run`
|
||||
* check tests pass/fail (by exitcode == 0)
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_BLINK=1 run`
|
||||
* GPIO on and off
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_UART_WRITE=1 run`
|
||||
* GPIO on/off and `counter: %` writes
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_IPC=1 run`
|
||||
* ASCII display draw
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_INPUT_DUMP=1 run` not implemented
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_QRCODE=1 run`
|
||||
* Some writes to display
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_DISPLAY=1 run`
|
||||
* Some writes to display
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=local APP_EXAMPLE_FATFS=1 flash`
|
||||
* TODO: FatFs emulation and test not implemented
|
||||
|
||||
F2 target:
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_TEST=1 clean`
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_TEST=1 flash`
|
||||
* check UART for test pass/fail
|
||||
* blue led when test is running
|
||||
* green led if test is passed
|
||||
* red led if test is failed
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_BLINK=1 flash`
|
||||
* Red LED blink (1s period)
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_UART_WRITE=1 flash`
|
||||
* Red LED shortly blinking, `counter: %` writes to UART
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_IPC=1 flash`
|
||||
* ASCII display draw in UART
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_INPUT_DUMP=1 flash`
|
||||
* Press all buttons, `state` and `event` writes to UART
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_QRCODE=1 flash`
|
||||
* QR code show on the screen
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_DISPLAY=1 flash`
|
||||
* `Hello world` show on the screen
|
||||
|
||||
* `docker-compose exec dev make -C firmware TARGET=f2 APP_EXAMPLE_FATFS=1 flash`
|
||||
* `Init sd card error` on the screen
|
||||
* Insert SD-card
|
||||
* Reboot
|
||||
* Show file list on the screen
|
||||
* Scroll by pressing up and down
|
||||
22
wiki/testing/Input-test.md
Normal file
22
wiki/testing/Input-test.md
Normal file
@@ -0,0 +1,22 @@
|
||||
1. `docker-compose exec dev make -C target_f2 example_input_dump`
|
||||
2. Flash
|
||||
3. For x in ```
|
||||
[
|
||||
(Up, "00"),
|
||||
(Down, "01"),
|
||||
(Right, "02"),
|
||||
(Left, "03"),
|
||||
(Ok, "04"),
|
||||
(Back, "05"),
|
||||
]
|
||||
```
|
||||
* Press ${x[0]}
|
||||
* wait 0.05
|
||||
* Expect: Uart: "event: ${x[1]} pressed"
|
||||
* wait 0.05
|
||||
* Release ${x[0]}
|
||||
* wait 0.05
|
||||
* Expect: Uart: "event: ${x[1]} released"
|
||||
* wait 0.05
|
||||
|
||||
TODO: add debouncing check (multiple press and check there is no multiple events)
|
||||
Reference in New Issue
Block a user