liz/veilid
liz/veilid
1
0
Fork 0
Code Pull Requests Packages Releases Activity
1,434 Commits 1 Branch 0 Tags 56 MiB
353c907497
Commit Graph

696 Commits

Author SHA1 Message Date
Christien Rioux
d68a543800 fix ios 2023-08-29 17:50:25 -05:00
Christien Rioux
e302b764d0 docs and tests work 2023-08-29 15:15:47 -05:00
Christien Rioux
3125c19f02 doc work 2023-08-27 16:39:50 -05:00
Christien Rioux
8c366387eb Merge branch 'fix-set-record-data-size-call' into 'main' 
call set_record_data_size with accumulated size

See merge request veilid/veilid!155
 2023-08-27 16:07:38 +00:00
Christien Rioux
59dda0febe Merge branch 'api_startup_json_skip_all' into 'main' 
api_startup_json: use 'skip_all'

See merge request veilid/veilid!154
 2023-08-27 15:58:00 +00:00
Rivka Segan
5dd0a3793b call set_record_data_size with accumulated size 
set_subkey corrupts record_data_size in a Record struct by calling
set_record_data_size with a value that depends only on the length of
the new subkey value. This leads to various undesirable outcomes, such
as: applications can write more than MAX_RECORD_DATA_SIZE without
encountering the intended "veilid.error.VeilidAPIErrorGeneric:
label='Generic' message='dht record too large'" error message, and
"panicked at 'attempt to subtract with overflow'" (i.e., an attempt to
set a negative value of a size) if a subkey's new length is less than
a subkey's old length. Typically, record_data_size in a Record struct
will be incorrect if a value was set for more than one subkey. Some
users might want to start over with a table_store that doesn't have
any incorrect record_data_size values.

The issue begins here:
6f71c6a00a/veilid-core/src/storage_manager/record_store.rs (L583-L586)

and is triggered here:
6f71c6a00a/veilid-core/src/storage_manager/record_store.rs (L613-L615)

It should be clear that new_record_data_size is only related to the
subkey that is currently being set. The amount of data in the record,
before set_subkey is called, is ignored. It appears that
new_total_size, not new_record_data_size, was intended to be used for
set_record_data_size, and this change succeeds for me in limited
testing but I don't have a comprehensive test suite.

One way to reproduce is by running the code from
https://gitlab.com/vatueil/veilid-file on a greater than 1 MB file
while watching variable values within
veilid-core/src/storage_manager/record_store.rs. For example: "poetry
run file put /usr/bin/tcpdump" (1.3 MB on Ubuntu 23.04). With the
original Veilid code, each of the dozens of subkey writes is checking
whether a roughly 32K value is greater than 1048576, it never is, and
thus there is never a "dht record too large" error. With the patch in
this MR, each of the dozens of subkey writes is checking whether an
ever-increasing value is greater than 1048576, it eventually is, and
the "dht record too large" error is printed. With the patch, one can
work with smaller files, e.g., do "poetry run file put /usr/bin/ssh"
(0.8 MB) followed by "poetry run file get VLD0:<_insert_key_here_>
ssh-copy" and the retrieved file ssh-copy is identical to
/usr/bin/ssh.

The more detailed behavior is that the modified code has
record.total_size of 350 on the first iteration, then 33596, 66842,
100088, etc. The original code also has record.total_size of 350 on
the first iteration, but then stays at 33246 forever (33246 is the
user-supplied subkey size of 32768, plus 128, plus the minimum record
size of 350),
 2023-08-26 07:08:47 +00:00
Bruno Bigras
ff6f04be8c api_startup_json: use 'skip_all' 2023-08-26 02:01:57 +00:00
Christien Rioux
248b21a951 proper relay switch, fix wasm 2023-08-24 18:59:33 -04:00
John Smith
6bb35dd6a6 skip publishing relay for fullconenat 2023-08-24 18:35:37 -04:00
John Smith
945215aba1 speed up public address detection 2023-08-24 18:35:37 -04:00
John Smith
4eca53fd9b direct port forward detection 2023-08-24 18:35:37 -04:00
Christien Rioux
654b5dfebc simplify code 2023-08-24 18:35:37 -04:00
Christien Rioux
34c686c227 xfer 2023-08-24 18:35:37 -04:00
Christien Rioux
030a0073da fix public address check 2023-08-24 18:35:37 -04:00
Christien Rioux
1315766fa9 eliminate network keying from bootstrap name 2023-08-22 15:11:45 -04:00
Christien Rioux
e504da2564 xfer 2023-08-22 15:11:45 -04:00
Christien Rioux
0249b7c7ae dial info failure reprioritization 2023-08-22 15:11:45 -04:00
Christien Rioux
10ec693fb4 Merge branch 'fix/value-data-api-crash' into 'main' 
fix: large value_data length in api crashes server

See merge request veilid/veilid!139
 2023-08-22 19:00:50 +00:00
Cheradenine Zakalwe
0ce19d85fa fix: large value_data length in api crashes server 2023-08-22 19:00:49 +00:00
Christien Rioux
eb4f29900d Merge branch 'mr-test' into 'main' 
Fix a couple of minor speedup issues

See merge request veilid/veilid!142
 2023-08-22 18:58:30 +00:00
a1ecbr0wn
3f9f31a0a2
Fix a couple of minor speedup issues 2023-08-22 13:00:53 +01:00
pbarry25
0b018edfe5
Fix minor typos 2023-08-21 20:12:20 -05:00
John Smith
f43462da8c add more changelog and clean up test output 2023-08-20 12:10:54 -04:00
John Smith
2d454c2654 fix comment 2023-08-20 11:52:54 -04:00
John Smith
c1cbcbe7c1 switch back to Sha512 for ed25519 compliance. add domain separation for signing and crypt 2023-08-20 11:47:29 -04:00
John Smith
d0fabbe3d9 Merge branch 'main' of gitlab.com:veilid/veilid into windows-fix 2023-08-20 11:25:39 -04:00
John Smith
1a3c87f9f1 fix nul termination 2023-08-20 11:23:17 -04:00
John Smith
964741463e vld0 work 2023-08-20 10:54:26 -04:00
John Smith
a7860d829e dh fix 2023-08-20 01:37:49 -04:00
John Smith
909a2f5362 windows fixess 2023-08-19 21:21:58 -04:00
Imuli
8f18569e30
use strict verification for ed25519 
This enables all the signature malleability checks (including one that
is not RFC8032 compliant but are still likely important for Veilid.)

For details on the additional check, see
https://docs.rs/ed25519-dalek/2.0.0/ed25519_dalek/struct.VerifyingKey.html#method.verify_strict
 2023-08-19 23:35:34 +00:00
John Smith
a9c13d45fd add changelog and missing file 2023-08-19 18:24:25 -04:00
John Smith
eae839e484 decompression fix for OOM 2023-08-19 18:13:30 -04:00
John Smith
89b357350d upgrade dependencies and fix cargo locks 2023-08-19 15:16:51 -04:00
Christien Rioux
2c779b2257 fanout debugging 2023-08-18 00:06:21 -04:00
Christien Rioux
b2503ae789 offline work 2023-08-18 00:06:21 -04:00
Christien Rioux
435469ce94 offline subkey writes 2023-08-18 00:06:21 -04:00
Christien Rioux
8e1ed1e3f1 fix crash 2023-08-18 00:06:21 -04:00
Christien Rioux
ef327fb963 crash fix 2023-08-18 00:06:21 -04:00
Christien Rioux
559ac5f162 fix server port allocation 2023-08-18 00:06:21 -04:00
Christien Rioux
79bf6fca69 fix bug 2023-08-18 00:06:21 -04:00
Christien Rioux
01aa411498 punishments 2023-08-18 00:06:21 -04:00
Christien Rioux
ffdd885aa4 test 2023-07-30 17:20:36 -04:00
John Smith
422a645708 python and api fixes 2023-07-30 16:45:20 -04:00
John Smith
9551ecd9a3 fix compilation error 2023-07-30 16:25:17 -04:00
John Smith
079211890b fix lru bug in connection table 2023-07-30 16:15:57 -04:00
Christien Rioux
91fab6ce5a Merge branch 'veilidchat-work' into 'main' 2023-07-25 17:22:44 +00:00
Teknique
7fa1df0474 Check b.len() < Self::FIXED_SIZE, not b.len() < 4 
The original thinking was that if len(b) < FIXED_SIZE, then that would
be picked up later by the "invalid member length" check. In that case,
this only really *needs* to make sure that the check after this for
"wrong fourcc" wouldn't fail. But if len(b) < FIXED_SIZE, it really is
an invalid size, and should get that error message before even starting
to validate its other qualities.
 2023-07-25 08:39:15 -07:00
Christien Rioux
00aad2c728 bugfixes 2023-07-25 01:04:22 -04:00
Teknique
f8bb97b39c Fix(?) size check in try_from for DHTSchemaSMPL 2023-07-24 21:43:14 -07:00
Christien Rioux
a6666d3a6c addresses 2023-07-23 23:13:42 -04:00
Christien Rioux
b993063aba network and ios fixes 2023-07-23 21:49:10 -04:00
Christien Rioux
cff955782b fix fox nodes length issue 2023-07-22 15:37:15 -04:00
Christien Rioux
9d3e847a68 more punishment cleanup 2023-07-21 14:30:10 -04:00
Christien Rioux
3f59f3bde3 cleanup warning 2023-07-21 09:48:30 -04:00
Christien Rioux
674a4d26f3 attempt to fix suspend issue 2023-07-21 09:44:14 -04:00
Christien Rioux
3224a315c3 proper node info filter for fanout 2023-07-20 17:52:45 -04:00
Christien Rioux
dfc2a09847 relay work 2023-07-19 20:55:37 -04:00
Christien Rioux
8d4f9cebab outbound relay fix 2023-07-19 14:45:05 -04:00
John Smith
9d0479fc45 better peer minimum refresh 2023-07-19 12:32:03 -04:00
Christien Rioux
f65400a1ce network fixes 2023-07-19 10:07:51 -04:00
Christien Rioux
05c75a5933 more error detail 2023-07-18 10:59:43 -04:00
John Smith
7658e2cb42 fix punishment 2023-07-17 17:51:05 -04:00
John Smith
304e9fd117 nodeinfo update fix 2023-07-16 13:29:41 -04:00
John Smith
f3d99541cc ws err 2023-07-16 12:56:02 -04:00
John Smith
3b76b1f81f network fixes 2023-07-16 12:28:27 -04:00
John Smith
823db3adf2 reverse connect fix 2023-07-16 00:21:19 -04:00
Christien Rioux
befb100ba4 compress envelopes with lz4 2023-07-15 21:44:36 -04:00
Christien Rioux
21ecd64ff8 loosen up on dropped rpcs 2023-07-15 20:03:58 -04:00
Christien Rioux
3264b568d0 punish by node id 2023-07-15 19:35:22 -04:00
Christien Rioux
80cb23c0c6 remove rkyv 2023-07-15 19:35:22 -04:00
Christien Rioux
19f384ab33 refactor create dht value 2023-07-15 19:35:22 -04:00
Christien Rioux
4960d13447 more semantics 2023-07-14 19:37:06 -04:00
Christien Rioux
e61d6be6a9 allow revert to invalid nodeinfo 2023-07-14 19:14:12 -04:00
Christien Rioux
167374969d connections work 2023-07-14 17:13:58 -04:00
Christien Rioux
6c2aaa16c6 relay work 2023-07-14 16:54:29 -04:00
John Smith
41b9a22595 fix keepalives 2023-07-14 14:21:00 -04:00
Christien Rioux
742b8e09a5 revert binding change 2023-07-14 00:24:19 -04:00
Christien Rioux
aec9f60290 ipv6 nat 2023-07-13 23:22:37 -04:00
Christien Rioux
858b0e6617 use unbound connections for outbound 2023-07-13 22:31:47 -04:00
Christien Rioux
73498c991f fix dns error 2023-07-13 21:44:34 -04:00
John Smith
7a737abb12 simplify tracing 2023-07-13 21:12:59 -04:00
Christien Rioux
0f33dfb15d more network triage 2023-07-13 19:54:08 -04:00
Christien Rioux
70a0346cc3 fix some more network issues 2023-07-13 18:52:03 -04:00
Christien Rioux
5977b6a141 add more error telemetry 2023-07-13 14:16:20 -04:00
John Smith
0a16938f9b fix signal 2023-07-12 20:12:45 -04:00
John Smith
714eca1411 relay fixes 2023-07-12 19:38:35 -04:00
John Smith
87694f5f4a more entries detail 2023-07-12 00:04:44 -04:00
John Smith
2a807715b6 network debugging 2023-07-11 21:56:13 -04:00
John Smith
c16888e214 fix missing relay issue 2023-07-04 15:34:48 -05:00
John Smith
b5906a52bc fix wasm 2023-07-04 15:34:48 -05:00
John Smith
907075411d protocol level capabilities 2023-07-04 12:35:48 -04:00
John Smith
8f721c428b more capability work 2023-07-04 00:24:55 -04:00
John Smith
e674eaf496 capability work 2023-07-03 18:01:02 -04:00
John Smith
6de2ccb1f9 more punish fixes 2023-07-03 16:42:49 -04:00
John Smith
ffc54f482e more capabilities 2023-07-03 15:56:13 -04:00
John Smith
dfb4eefd92 switch out capabilities 2023-07-03 15:10:28 -04:00
John Smith
cf9d8cf7c2 fix punish 2023-07-03 12:20:39 -04:00
John Smith
b7e531f35b fix blocked issue 2023-07-03 11:38:40 -04:00
John Smith
cb66af7df3 address filter 2023-07-02 00:17:04 -04:00
John Smith
d290a66f32 dht fixes 2023-07-01 10:45:31 -04:00
John Smith
176d9ac68e dht testing work 2023-06-29 22:18:45 -04:00
John Smith
d044f646bf fix serialize for keypair 2023-06-29 20:49:15 -04:00
John Smith
98a20f5921 fix attachment update 2023-06-28 23:46:29 -04:00
John Smith
05a9ee754e cleanup dht stuff and client/server api 2023-06-28 23:15:06 -04:00
John Smith
b01fb20ec9 add dht record get 2023-06-28 11:55:37 -04:00
John Smith
f1292694a2 enable full safety selection through api 2023-06-28 11:40:02 -04:00
John Smith
fde70610cc oop 2023-06-27 19:05:50 -04:00
John Smith
73c37aa4ca clean up valuesubkeyrangeseT 2023-06-27 18:26:53 -04:00
John Smith
291e3ef2fe add better dht debugging 2023-06-26 21:29:02 -04:00
John Smith
62aeec6faf debug less 2023-06-25 18:28:32 -04:00
John Smith
c3639fd331 fix nodecontactmethod cache 2023-06-25 17:59:11 -04:00
John Smith
297908796c remove owo colors 2023-06-25 14:09:22 -04:00
John Smith
0f3e7010f2 sequencing 2023-06-25 01:53:06 -04:00
John Smith
3e23f808d0 more sequencing 2023-06-25 01:23:24 -04:00
John Smith
234f048241 simplify tracing 2023-06-24 22:59:51 -04:00
John Smith
c8fdded5a7 fix crypto 
s
 2023-06-24 21:23:48 -04:00
John Smith
b1df2c9d2d fix traces 2023-06-24 20:29:36 -04:00
John Smith
909ef6bf69 messages cleanup 2023-06-24 20:23:33 -04:00
John Smith
4a14cabeeb handle unreachable 2023-06-24 18:40:19 -04:00
John Smith
355040a3e4 refactor and change protocol selection for new connections 2023-06-24 18:12:17 -04:00
John Smith
197b7fef6e reliability work 2023-06-24 11:16:34 -04:00
John Smith
acebcb7947 network keying 2023-06-23 21:12:48 -04:00
John Smith
bc6421acf7 fixes 2023-06-23 17:01:52 -04:00
John Smith
296ca5cadb Merge branch 'main' of gitlab.hackers.town:veilid/veilid into dht-testing 2023-06-23 12:05:40 -04:00
John Smith
e4f97cfefa assembly buffer 2023-06-23 12:05:28 -04:00
John Smith
dfc22aee8e fix tests 2023-06-22 18:31:31 -04:00
John Smith
d21f580de2 Merge branch 'main' of gitlab.hackers.town:veilid/veilid into dht-testing 2023-06-22 17:51:58 -04:00
John Smith
addfd64473 xfer 2023-06-22 17:42:34 -04:00
John Smith
7dbd1e8b92 frag work 2023-06-22 16:54:01 -04:00
John Smith
3d3582e688 assemblybuffer work 2023-06-21 23:35:33 -04:00
John Smith
05d774d23b log cleanup 2023-06-21 14:56:21 -04:00
John Smith
754abf2135 fix binding issues 2023-06-21 13:40:12 -04:00
John Smith
2272b8e139 xfer 2023-06-21 10:02:16 -04:00
John Smith
1c8ecab2b6 cli fixes 2023-06-20 23:46:39 -04:00
John Smith
f17c2f62cb xfer 2023-06-19 22:35:49 -04:00
John Smith
ea651e526d xfer 2023-06-19 11:29:33 -04:00
John Smith
e2824da06e another tabledb fix 2023-06-18 21:34:40 -04:00
John Smith
2314fcb57e more release lifetime cleanup and base class contextmanager stuff 2023-06-18 20:57:51 -04:00
John Smith
b6e055e47d test work 2023-06-18 18:47:39 -04:00
John Smith
c278ddce53 fix udp hole punch goof 2023-06-16 22:28:04 -04:00
John Smith
bfb66e3000 extra error details in the event we see this again 2023-06-16 20:31:01 -04:00
John Smith
d308ebc324 fix tests 2023-06-16 14:23:36 -04:00
John Smith
9ffcc0da87 fix tests 2023-06-16 14:18:34 -04:00
John Smith
f3a3d5322c fixes for tests 2023-06-16 13:29:25 -04:00
John Smith
c6e2836b85 xfer 2023-06-16 13:14:43 -04:00
John Smith
fe502a3645 Merge branch 'main' of gitlab.hackers.town:veilid/veilid into json-rpc 2023-06-16 13:13:51 -04:00
John Smith
14ba85efda fixes 2023-06-16 11:57:55 -04:00
John Smith
d114ea3b72 crufy 2023-06-15 20:29:12 -04:00
John Smith
d6f442d431 better error handling 2023-06-15 20:22:54 -04:00