liz/veilid
liz/veilid
1
0
Fork 0
Code Pull Requests Packages Releases Activity

Merge branch 'veilid6/document-config' into 'main'

Browse Source 
veilid6, begin commenting config w/ rustdoc

See merge request veilid/veilid!7
This commit is contained in:
John Smith 2022-09-23 15:29:11 +00:00
commit 57f09cfee6
7 changed files with 581 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -61,4 +61,3 @@ $RECYCLE.BIN/
### Rust ### Rust
target/ target/
logs/ logs/

46
README.md
View File

@ -129,7 +129,9 @@ cd veilid-flutter
**TODO** **TODO**
## Veilid Server ## Running the Application(s)
### Veilid Server
In order to run the `veilid-server` locally: In order to run the `veilid-server` locally:
@ -144,7 +146,22 @@ In order to see what options are available:
cargo run -- --help cargo run -- --help
``` ```
## Veilid CLI #### Configuration
`veilid-server` has a wealth of configuration options. Further documentation on
the format of the `veilid-server.conf` file may be found [in the project /doc
directory](./doc/config/veilid-server-config.md).
When running `veilid-server` in a Unix-like environment, the application will
look for its config file under `/etc/veilid-server/`. If the config file is not
found in this location, `veilid-server` will follow the XDG user directory spec
and look in `~/.config/veilid-server`.
When running under Windows, the `veilid-server.conf` file may be created at
`C:\Users\<user>\AppData\Roaming\Veilid\Veilid\`, and when running under macOS,
at `/Users/<user>/Library/Application Support/org.Veilid.Veilid`.
### Veilid CLI
In order to connect to your local `veilid-server`: In order to connect to your local `veilid-server`:
@ -158,3 +175,28 @@ Similar to `veilid-server`, you may see CLI options by typing:
```shell ```shell
cargo run -- --help cargo run -- --help
``` ```
## Building the Application
### Linux Packages
Veilid server and cli can be built locally using the
[Earthly](https://earthly.dev/) framework. After [installing earthly on your
local machine](https://earthly.dev/get-earthly), you may use the `earthly` cli
to initiate a build:
```shell
earthly +package-linux
```
This will assemble all dependencies and build `.deb` packages for both amd64 and
arm64 platforms. Earthly, built on Docker, caches build layers, so after a
longer first build, subsequent builds should be much quicker.
During development, you may want to kick off specific build steps. To see a list
of the build steps configured, consult the `Earthfile`, or you may use the
`earthly` cli:
```shell
earthly ls
```

127
doc/config/sample.config Normal file
View File

@ -0,0 +1,127 @@
daemon:
enabled: false
client_api:
enabled: true
listen_address: 'localhost:5959'
auto_attach: true
logging:
system:
enabled: false
level: 'info'
terminal:
enabled: true
level: 'info'
file:
enabled: false
path: ''
append: true
level: 'info'
api:
enabled: true
level: 'info'
otlp:
enabled: false
level: 'trace'
grpc_endpoint: 'localhost:4317'
testing:
subnode_index: 0
core:
protected_store:
allow_insecure_fallback: true
always_use_insecure_storage: true
insecure_fallback_directory: '%INSECURE_FALLBACK_DIRECTORY%'
delete: false
table_store:
directory: '%TABLE_STORE_DIRECTORY%'
delete: false
block_store:
directory: '%BLOCK_STORE_DIRECTORY%'
delete: false
network:
connection_initial_timeout_ms: 2000
connection_inactivity_timeout_ms: 60000
max_connections_per_ip4: 32
max_connections_per_ip6_prefix: 32
max_connections_per_ip6_prefix_size: 56
max_connection_frequency_per_min: 128
client_whitelist_timeout_ms: 300000
reverse_connection_receipt_time_ms: 5000
hole_punch_receipt_time_ms: 5000
node_id: ''
node_id_secret: ''
bootstrap: ['bootstrap.dev.veilid.net']
bootstrap_nodes: []
routing_table:
limit_over_attached: 64
limit_fully_attached: 32
limit_attached_strong: 16
limit_attached_good: 8
limit_attached_weak: 4
rpc:
concurrency: 0
queue_size: 1024
max_timestamp_behind_ms: 10000
max_timestamp_ahead_ms: 10000
timeout_ms: 10000
max_route_hop_count: 7
dht:
resolve_node_timeout:
resolve_node_count: 20
resolve_node_fanout: 3
max_find_node_count: 20
get_value_timeout:
get_value_count: 20
get_value_fanout: 3
set_value_timeout:
set_value_count: 20
set_value_fanout: 5
min_peer_count: 20
min_peer_refresh_time_ms: 2000
validate_dial_info_receipt_time_ms: 2000
upnp: true
natpmp: false
detect_address_changes: true
enable_local_peer_scope: false
restricted_nat_retries: 0
tls:
certificate_path: '%CERTIFICATE_PATH%'
private_key_path: '%PRIVATE_KEY_PATH%'
connection_initial_timeout_ms: 2000
application:
https:
enabled: false
listen_address: ':5150'
path: 'app'
# url: 'https://localhost:5150'
http:
enabled: false
listen_address: ':5150'
path: 'app'
# url: 'http://localhost:5150'
protocol:
udp:
enabled: true
socket_pool_size: 0
listen_address: ':5150'
# public_address: ''
tcp:
connect: true
listen: true
max_connections: 32
listen_address: ':5150'
#'public_address: ''
ws:
connect: true
listen: true
max_connections: 16
listen_address: ':5150'
path: 'ws'
# url: 'ws://localhost:5150/ws'
wss:
connect: true
listen: false
max_connections: 16
listen_address: ':5150'
path: 'ws'
# url: ''

307
doc/config/veilid-server-config.md Normal file
View File

@ -0,0 +1,307 @@
---
title: Veilid Server Configuration
keywords:
- config
- veilid-server
status: Draft
---
# Veilid Server Configuration
## Configuration File
`veilid-server` may be run using configuration from both command-line arguments
and the `veilid-server.conf` file.
## Global Directives
| Directive | Description |
|-------------------------------|-----------------------------------------|
| [daemon](#daemon) | Run `veilid-server` in the background |
| [client\_api](#client_api) ||
| [auto\_attach](#auto_attach) ||
| [logging](#logging) ||
| [testing](#testing) ||
| [core](#core) ||
### daemon
```yaml
daemon:
enabled: false
```
### client_api
```yaml
client_api:
enabled: true
listen_address: 'localhost:5959'
```
| Parameter | Description |
|-----------------------------------------------|-------------|
| [enabled](#client_apienabled) ||
| [listen\_address](#client_apilisten_address) ||
#### client\_api:enabled
**TODO**
#### client\_api:listen\_address
**TODO**
### auto\_attach
```yaml
auto_attach: true
```
### logging
```yaml
logging:
system:
enabled: false
level: 'info'
terminal:
enabled: true
level: 'info'
file:
enabled: false
path: ''
append: true
level: 'info'
api:
enabled: true
level: 'info'
otlp:
enabled: false
level: 'trace'
grpc_endpoint: 'localhost:4317'
```
| Parameter | Description |
|-------------------------------|-------------|
| [system](#loggingsystem) ||
| [terminal](#loggingterminal) ||
| [file](#loggingfile) ||
| [api](#loggingapi) ||
| [otlp](#loggingotlp) ||
#### logging:system
```yaml
system:
enabled: false
level: 'info'
```
#### logging:terminal
```yaml
terminal:
enabled: true
level: 'info'
```
#### logging:file
```yaml
file:
enabled: false
path: ''
append: true
level: 'info'
```
#### logging:api
```yaml
api:
enabled: true
level: 'info'
```
#### logging:otlp
```yaml
otlp:
enabled: false
level: 'trace'
grpc_endpoint: 'localhost:4317'
```
### testing
```yaml
testing:
subnode_index: 0
```
### core
| Parameter | Description |
|-------------------------------------------|-------------|
| [protected\_store](#coreprotected_store) ||
| [table\_store](#coretable_store) ||
| [block\_store](#block_store) ||
| [network](#corenetwork) ||
#### core:protected\_store
```yaml
protected_store:
allow_insecure_fallback: true
always_use_insecure_storage: true
insecure_fallback_directory: '%INSECURE_FALLBACK_DIRECTORY%'
delete: false
```
#### core:table\_store
```yaml
table_store:
directory: '%TABLE_STORE_DIRECTORY%'
delete: false
```
#### core:block\_store
```yaml
block_store:
directory: '%BLOCK_STORE_DIRECTORY%'
delete: false
```
#### core:network
```yaml
network:
connection_initial_timeout_ms: 2000
connection_inactivity_timeout_ms: 60000
max_connections_per_ip4: 32
max_connections_per_ip6_prefix: 32
max_connections_per_ip6_prefix_size: 56
max_connection_frequency_per_min: 128
client_whitelist_timeout_ms: 300000
reverse_connection_receipt_time_ms: 5000
hole_punch_receipt_time_ms: 5000
node_id: ''
node_id_secret: ''
bootstrap: ['bootstrap.dev.veilid.net']
bootstrap_nodes: []
upnp: true
natpmp: false
detect_address_changes: true
enable_local_peer_scope: false
restricted_nat_retries: 0
```
| Parameter | Description |
|---------------------------------------------|-------------|
| [routing\_table](#corenetworkrouting_table) ||
| [rpc](#corenetworkrpc) ||
| [dht](#corenetworkdht) ||
| [tls](#corenetworktls) ||
| [application](#corenetworkapplication) ||
| [protocol](#corenetworkprotocol) ||
#### core:network:routing\_table
```yaml
routing_table:
limit_over_attached: 64
limit_fully_attached: 32
limit_attached_strong: 16
limit_attached_good: 8
limit_attached_weak: 4
```
#### core:network:rpc
```yaml
rpc:
concurrency: 0
queue_size: 1024
max_timestamp_behind_ms: 10000
max_timestamp_ahead_ms: 10000
timeout_ms: 10000
max_route_hop_count: 7
```
#### core:network:dht
```yaml
dht:
resolve_node_timeout:
resolve_node_count: 20
resolve_node_fanout: 3
max_find_node_count: 20
get_value_timeout:
get_value_count: 20
get_value_fanout: 3
set_value_timeout:
set_value_count: 20
set_value_fanout: 5
min_peer_count: 20
min_peer_refresh_time_ms: 2000
validate_dial_info_receipt_time_ms: 2000
```
#### core:network:tls
```yaml
tls:
certificate_path: '%CERTIFICATE_PATH%'
private_key_path: '%PRIVATE_KEY_PATH%'
connection_initial_timeout_ms: 2000
```
#### core:network:application
```yaml
application:
https:
enabled: false
listen_address: ':5150'
path: 'app'
# url: 'https://localhost:5150'
http:
enabled: false
listen_address: ':5150'
path: 'app'
# url: 'http://localhost:5150'
```
#### core:network:protocol
```yaml
protocol:
udp:
enabled: true
socket_pool_size: 0
listen_address: ':5150'
# public_address: ''
tcp:
connect: true
listen: true
max_connections: 32
listen_address: ':5150'
#'public_address: ''
ws:
connect: true
listen: true
max_connections: 16
listen_address: ':5150'
path: 'ws'
# url: 'ws://localhost:5150/ws'
wss:
connect: true
listen: false
max_connections: 16
listen_address: ':5150'
path: 'ws'
# url: ''
```

92
veilid-core/src/veilid_config.rs
View File

@ -6,6 +6,16 @@ use serde::*;
pub type ConfigCallbackReturn = Result<Box<dyn core::any::Any + Send>, VeilidAPIError>; pub type ConfigCallbackReturn = Result<Box<dyn core::any::Any + Send>, VeilidAPIError>;
pub type ConfigCallback = Arc<dyn Fn(String) -> ConfigCallbackReturn + Send + Sync>; pub type ConfigCallback = Arc<dyn Fn(String) -> ConfigCallbackReturn + Send + Sync>;
/// Enable and configure HTTPS access to the Veilid node
///
/// ```yaml
/// https:
/// enabled: false
/// listen_address: ':5150'
/// path: 'app'
/// url: 'https://localhost:5150'
/// ```
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigHTTPS { pub struct VeilidConfigHTTPS {
pub enabled: bool, pub enabled: bool,
@ -14,6 +24,16 @@ pub struct VeilidConfigHTTPS {
pub url: Option<String>, // Fixed URL is not optional for TLS-based protocols and is dynamically validated pub url: Option<String>, // Fixed URL is not optional for TLS-based protocols and is dynamically validated
} }
/// Enable and configure HTTP access to the Veilid node
///
/// ```yaml
/// http:
/// enabled: false
/// listen_address: ':5150'
/// path: 'app"
/// url: 'https://localhost:5150'
/// ```
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigHTTP { pub struct VeilidConfigHTTP {
pub enabled: bool, pub enabled: bool,
@ -22,12 +42,28 @@ pub struct VeilidConfigHTTP {
pub url: Option<String>, pub url: Option<String>,
} }
/// Application configuration
///
/// Configure web access to the Prograssive Web App (PWA)
///
/// To be implemented...
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigApplication { pub struct VeilidConfigApplication {
pub https: VeilidConfigHTTPS, pub https: VeilidConfigHTTPS,
pub http: VeilidConfigHTTP, pub http: VeilidConfigHTTP,
} }
/// Enable and configure UDP
///
/// ```yaml
/// udp:
/// enabled: true
/// socket_pool_size: 0
/// listen_address: ':5150'
/// public_address: ''
/// ```
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigUDP { pub struct VeilidConfigUDP {
pub enabled: bool, pub enabled: bool,
@ -36,6 +72,16 @@ pub struct VeilidConfigUDP {
pub public_address: Option<String>, pub public_address: Option<String>,
} }
/// Enable and configure TCP
///
/// ```yaml
/// tcp:
/// connect: true
/// listen: true
/// max_connections: 32
/// listen_address: ':5150'
/// public_address: ''
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigTCP { pub struct VeilidConfigTCP {
pub connect: bool, pub connect: bool,
@ -45,6 +91,17 @@ pub struct VeilidConfigTCP {
pub public_address: Option<String>, pub public_address: Option<String>,
} }
/// Enable and configure Web Sockets
///
/// ```yaml
/// ws:
/// connect: true
/// listen: true
/// max_connections: 16
/// listen_address: ':5150'
/// path: 'ws'
/// url: 'ws://localhost:5150/ws'
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigWS { pub struct VeilidConfigWS {
pub connect: bool, pub connect: bool,
@ -55,6 +112,17 @@ pub struct VeilidConfigWS {
pub url: Option<String>, pub url: Option<String>,
} }
/// Enable and configure Secure Web Sockets
///
/// ```yaml
/// wss:
/// connect: true
/// listen: false
/// max_connections: 16
/// listen_address: ':5150'
/// path: 'ws'
/// url: ''
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigWSS { pub struct VeilidConfigWSS {
pub connect: bool, pub connect: bool,
@ -65,6 +133,13 @@ pub struct VeilidConfigWSS {
pub url: Option<String>, // Fixed URL is not optional for TLS-based protocols and is dynamically validated pub url: Option<String>, // Fixed URL is not optional for TLS-based protocols and is dynamically validated
} }
/// Configure Network Protocols
///
/// Veilid can communicate over UDP, TCP, and Web Sockets.
///
/// All protocols are available by default, and the Veilid node will
/// sort out which protocol is used for each peer connection.
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigProtocol { pub struct VeilidConfigProtocol {
pub udp: VeilidConfigUDP, pub udp: VeilidConfigUDP,
@ -73,6 +148,14 @@ pub struct VeilidConfigProtocol {
pub wss: VeilidConfigWSS, pub wss: VeilidConfigWSS,
} }
/// Configure TLS
///
/// ```yaml
/// tls:
/// certificate_path: /path/to/cert
/// private_key_path: /path/to/private/key
/// connection_initial_timeout_ms: 2000
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigTLS { pub struct VeilidConfigTLS {
pub certificate_path: String, pub certificate_path: String,
@ -80,6 +163,8 @@ pub struct VeilidConfigTLS {
pub connection_initial_timeout_ms: u32, pub connection_initial_timeout_ms: u32,
} }
/// Configure the Distributed Hash Table (DHT)
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigDHT { pub struct VeilidConfigDHT {
pub resolve_node_timeout_ms: Option<u32>, pub resolve_node_timeout_ms: Option<u32>,
@ -97,6 +182,8 @@ pub struct VeilidConfigDHT {
pub validate_dial_info_receipt_time_ms: u32, pub validate_dial_info_receipt_time_ms: u32,
} }
/// Configure RPC
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigRPC { pub struct VeilidConfigRPC {
pub concurrency: u32, pub concurrency: u32,
@ -107,6 +194,8 @@ pub struct VeilidConfigRPC {
pub max_route_hop_count: u8, pub max_route_hop_count: u8,
} }
/// Configure the network routing table
///
#[derive(Default, Debug, Clone, Serialize, Deserialize)] #[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub struct VeilidConfigRoutingTable { pub struct VeilidConfigRoutingTable {
pub limit_over_attached: u32, pub limit_over_attached: u32,
@ -244,6 +333,9 @@ pub struct VeilidConfigInner {
pub network: VeilidConfigNetwork, pub network: VeilidConfigNetwork,
} }
/// The Veilid Configuration
///
/// Veilid is configured
#[derive(Clone)] #[derive(Clone)]
pub struct VeilidConfig { pub struct VeilidConfig {
inner: Arc<RwLock<VeilidConfigInner>>, inner: Arc<RwLock<VeilidConfigInner>>,

10
veilid-server/src/settings.rs
View File

@ -752,6 +752,16 @@ impl Settings {
Ok(()) Ok(())
} }
/// Determine default config path
///
/// In a unix-like environment, veilid-server will look for its config file
/// in /etc/veilid-server. If a config is not found in this location, it will
/// follow the XDG user directory spec, and look in `~/.config/veilid-server/`.
///
/// For Windows, a user-local config may be created at
/// `C:\Users\<user>\AppData\Roaming\Veilid\Veilid`, and for macOS, at
/// `/Users/<user>/Library/Application Support/org.Veilid.Veilid`
///
pub fn get_default_config_path() -> PathBuf { pub fn get_default_config_path() -> PathBuf {
#[cfg(unix)] #[cfg(unix)]
{ {

1
veilid-server/src/unix.rs
View File

@ -26,6 +26,7 @@ async fn handle_signals(mut signals: Signals) {
} }
} }
#[warn(missing_docs)]
#[instrument(err)] #[instrument(err)]
pub fn run_daemon(settings: Settings, _matches: ArgMatches) -> EyreResult<()> { pub fn run_daemon(settings: Settings, _matches: ArgMatches) -> EyreResult<()> {
let daemon = { let daemon = {