Commit Graph

354 Commits

Author SHA1 Message Date
Ske
7488ca459b Hopefully fix shard info tracking (again) 2020-05-01 20:27:51 +02:00
Ske
91b60367f0 Hopefully fix error in LogClean module 2020-05-01 19:41:15 +02:00
Ske
e7b28261b2 "Properly" handle CloudFlare errors in webhook execution 2020-05-01 19:21:02 +02:00
Ske
48833639e2 Update shard status tracker for new library 2020-05-01 17:30:12 +02:00
Ske
d0f1aeb548 Fix member lookup in SanitizeEveryoneMaybe 2020-05-01 17:03:43 +02:00
Ske
e133bd657d Remove log channels if they don't exist 2020-05-01 16:36:21 +02:00
Ske
e4ebd2a5fe Merge branch 'refactor/dsharpplus'
It's heeeeeeeere~
2020-05-01 15:21:55 +02:00
Ske
546cb7f97a Remove webhook rate limit cache
The move to DSharpPlus makes it unnecessary, as D#+ can actually do webhook invocations on its own.
2020-05-01 00:00:33 +02:00
Ske
042327d4aa Add online user counting back (ish) 2020-04-30 23:55:12 +02:00
Ske
9fa36b78ff Fix the garish default D#+ colors 2020-04-29 00:25:01 +02:00
Ske
35e30f481b Fix error in stats collection task 2020-04-29 00:04:53 +02:00
Fennel
ebaded4bbd Fix Proxied Messages Ignored 2020-04-28 22:30:11 +02:00
Ske
5386f24b69 Fix permissions check when proxying 2020-04-22 09:27:55 +02:00
Ske
23cf06df4c Port some things, still does not compile 2020-04-17 23:10:01 +02:00
Ske
02e2c9314b Update webhook rate limit code 2020-04-16 22:30:13 +02:00
Ske
05595329b5 Collect metrics about webhook rate limit cache size 2020-04-16 18:20:53 +02:00
Ske
93bf7c6543 Add periodic garbage collection of webhook rate limit cache 2020-04-16 18:18:08 +02:00
Ske
fa70df8f98 Add proper webhook rate limit support 2020-03-27 00:01:42 +01:00
Ske
20789411f6 Cancel early when rate limited 2020-03-08 11:06:08 +01:00
Ske
f8d0fb2f4b Properly ignore invalid server-side data when executing webhooks 2020-03-08 10:55:47 +01:00
Ske
35b0659d05 Don't ping by reaction if sender doesn't have permissions 2020-03-04 18:39:51 +01:00
Ske
e49f2850c6 Add support for Vanessa log cleaning 2020-03-04 18:32:14 +01:00
Ske
ee3ac44664 Properly handle 5xx errors from webhook calls 2020-02-26 19:47:30 +01:00
Ske
eefbf0630e Fix fuzzy log clean match error with multiple messages 2020-02-22 20:08:20 +01:00
Ske
8d3be79d33 Don't attempt to send log messages if the bot doesn't have permissions 2020-02-22 01:54:10 +01:00
Ske
de141d629b Fix newline rendering in embeds on iOS 2020-02-20 22:57:37 +01:00
Ske
456fe8f7af Ignore weirdly formatted server errors too 2020-02-19 01:00:23 +01:00
Ske
55671b45d6 Add support for UnbelievaBoat 2020-02-15 14:17:25 +01:00
Ske
ccf81b0826 Add support for Pancake 2020-02-15 14:17:25 +01:00
Ske
e9cc8ed424 Add logger bot cleanup support 2020-02-15 14:17:25 +01:00
Ske
e70236b9c0 Send member card on proxy query 2020-02-15 00:12:46 +01:00
Ske
48342a2890 Fix spurious empty-string-description errors 2020-02-12 23:18:31 +01:00
Ske
dc10a8411a Disable autoproxy retriggering on edit 2020-02-12 17:44:24 +01:00
Ske
d0d3579b17 Add server-specific member avatars 2020-02-12 17:42:12 +01:00
Ske
6d5004bf54 Large refactor and project restructuring 2020-02-12 15:16:19 +01:00
Ske
1d3acb2392 Fix error when trying to DM a user with DMs disabled 2020-02-12 14:22:15 +01:00
Ske
1386e6743b Proxy edited messages if the message is the last one in the channel, and the edit introduces proxy tags where there were none previously 2020-02-12 14:21:48 +01:00
Ske
9d0af7f22a Fix image proxying with empty messages 2020-02-09 15:27:52 +01:00
Ske
3ce1078dc6 Add different text for systems with no members 2020-02-05 23:43:30 +01:00
Ske
f27e864731 Fix autoproxying someone else's bare tags 2020-02-03 16:10:43 +01:00
Ske
6863da9462 Disable Discord member caching entirely 2020-02-01 15:00:36 +01:00
Ske
82dfe43d5a Rework caching
This does a *lot* of things. Essentially, it replaces the existing
individual proxy- and autoproxy caches on the bot end with a
global cache (in Core) that handles all the caching at once, and
automatically invalidates the cache once something changes in the
datastore.

This allows us to do proxying and autoproxying with *zero database
queries* (best-case).
2020-02-01 14:40:57 +01:00
Ske
3ba6a115f0 Correct autoproxy escape character 2020-01-25 17:40:41 +01:00
Ske
83cfb3eb46 Add autoproxy functionality 2020-01-24 20:28:48 +01:00
Ske
b347d2d557 Add front history pagination; upgrade more store methods 2020-01-18 00:58:35 +01:00
Ske
8a689ac0f2 Upgrade various store methods to IAsyncEnumerable 2020-01-18 00:02:17 +01:00
Ske
98613c4287 Add system and member privacy support 2020-01-11 16:51:13 +01:00
Ske
cb1c44a27f Bail early if we don't have send message permission when proxying 2020-01-08 12:16:27 +01:00
Ske
73b2631280 Optimize database queries for proxy filtering 2019-12-28 12:00:52 +01:00
Ske
4d07886ec8 Add server-specific display names 2019-12-26 20:39:47 +01:00
Ske
633627c30a Fix image-only proxies where the tags have trailing/leading spaces 2019-12-26 19:19:06 +01:00
Ske
5ff32ea6b4 Re-enable proxy attachments 2019-12-23 13:55:43 +01:00
Ske
58cdb04ab9 Practice better disposal hygiene 2019-12-23 01:58:10 +01:00
Ske
3fdfaeab01 Disable message attachments as a temporary fix 2019-12-23 01:53:01 +01:00
Ske
affb42160a Oops pt. 3 2019-12-23 00:35:42 +01:00
Ske
de75752b5e Oops pt. 2 2019-12-23 00:35:25 +01:00
Ske
05282fd167 Reuse a singleton HttpClient for proxying 2019-12-23 00:29:04 +01:00
Ske
3c95f35553 Reduce timeouts on webhook invocations 2019-12-22 22:56:18 +01:00
Ske
cb836d5f7e Remove the ping reaction afterwards 2019-12-22 14:44:14 +01:00
Ske
639c813ce7 Add per-server per-system proxy enable toggle 2019-12-22 14:15:56 +01:00
Ske
857b0488b9 Add member pinging by reaction 2019-12-22 13:54:18 +01:00
Ske
bd8fb40bc7 Handle systems with (too) many linked accounts 2019-12-22 13:53:38 +01:00
Ske
992a37dd1b Lower logging level of proxy cache messages 2019-12-22 12:50:56 +01:00
Ske
b81eda47aa Expand pk;stats functionality 2019-12-22 12:50:47 +01:00
Ske
d42dea9e9f Allow multiple proxy attachments 2019-12-21 20:07:51 +01:00
Ske
474d561c54 Execute webhooks directly rather than through D.NET 2019-12-21 18:50:28 +01:00
Ske
378cba09e5 Add channel blacklists for logging and proxying
Necessary database migrations for this commit:
    alter table servers add column log_blacklist bigint[] not null default array[]::bigint[];
    alter table servers add column blacklist bigint[] not null default array[]::bigint[];
2019-11-03 19:15:50 +01:00
Ske
520e6f140c Fix front history lookup with very long switches 2019-11-03 00:46:38 +01:00
Ske
6c39b7549b Fix message querying actually for real 2019-10-31 21:14:01 +01:00
Ske
f4a5b409c3 Fix message querying for real this time 2019-10-31 16:36:33 +01:00
Ske
49dc25ee02 Allow keeping proxy tags when proxying messages.
Required database migration:

Closes #75.
2019-10-30 09:26:50 +01:00
Ske
b50c123ad4 Properly sort proxies before matching 2019-10-30 08:30:58 +01:00
Ske
393ee16c1b Add support for multiple proxy tags
Tangentially closes #103.
2019-10-30 08:18:59 +01:00
Ske
3c7afbfdaa Lower log level of an unnecessary webhook logging message 2019-10-28 17:03:16 +01:00
Ske
c639e9ba1e Include attachment image in message card 2019-10-28 17:02:59 +01:00
Ske
3369a06c4e Fix message querying on accounts with no roles 2019-10-28 16:50:41 +01:00
Ske
545e8df6a7 Reduce the log level of various common events 2019-10-27 23:44:27 +01:00
Ske
46672d264c Refactor log channel storage 2019-10-27 23:01:20 +01:00
Ske
6538c5d054 Fix line endings in WebhookCacheService 2019-10-27 13:09:07 +01:00
Ske
aed38f37f7 Allow starting bot before all shards are up 2019-10-27 13:01:33 +01:00
Ske
6a73b3bdd6 Refactor data stores, merging the Store classes 2019-10-26 19:45:44 +02:00
Ske
c4d28f64c7 Display system/member created date in embed footer.
Closes #122.
2019-10-18 13:29:16 +02:00
Astrid
1d7a4d78db
Merge pull request #126 from xBelladonna/feature/messagecard-nickname
Display guild user nickname on message card if set
2019-10-18 13:22:48 +02:00
Ske
1988b29fbc Refactor command system 2019-10-05 07:41:00 +02:00
xBelladonna
658bcf1faf Display guild user nickname on message card if set 2019-09-22 02:04:20 +09:30
Ske
0ec522ca0a Display member roles in message query card 2019-09-21 15:19:38 +02:00
Ske
9b0b8b4353 Fix Clyde, also without an attachment 2019-08-14 07:21:03 +02:00
Ske
74e0508065 Increase webhook name limit to 80 2019-08-14 07:16:48 +02:00
Ske
4e69531103 Ignore invalid colors in member card 2019-08-13 21:49:43 +02:00
Ske
8396e17a87 Attempt to fix webhook invocation errors 2019-08-12 16:38:34 +02:00
Ske
145ecb91ad Refactor proxy service 2019-08-12 05:47:55 +02:00
Ske
17c8b7b765 Properly retry client creation on failure 2019-08-12 04:42:16 +02:00
Ske
5b13c1b100 Cache the entire webhook client rather than just the token 2019-08-12 04:32:01 +02:00
Ske
1b32cd8b6d Log webhook send progress 2019-08-12 04:16:17 +02:00
Ske
aa04d2055e Avoid deleting messages from the store if they're not webhooks 2019-08-12 03:51:54 +02:00
Ske
423d23faf7 Cache account lookup in memory when proxying 2019-08-12 03:48:08 +02:00
Ske
a689d61788 Collect webhook response time and shard latency 2019-08-12 00:57:23 +02:00
Ske
90c6869b00 Collect statistics about shard status 2019-08-12 00:35:30 +02:00
Ske
942022d408 Track database handles over metrics 2019-08-11 22:56:20 +02:00
Bella | Nightshade
9f2faeea5a Escape markdown in proxy tags on member card (#118)
* Fix a typo

* Escape markdown in proxy tags on member card

* Whoops hhh

Added API documentation for POST to member endpoint
2019-08-09 12:55:15 +02:00
Ske
606f48fdb5 Add display name to member card 2019-08-09 10:39:03 +02:00
Ske
200ba6d84c Add member proxy display names 2019-08-09 10:12:38 +02:00
xBelladonna
fabf772085 Return original message ID to log channel
Logs the original (deleted) message's ID in addition to the proxied message's ID to the log channel if configured
2019-08-08 11:01:36 +02:00
Ske
59dbc5368b Filter everyone/here pings if the proxy sender doesn't have permission 2019-07-28 15:38:41 +02:00
Ske
defbf37cfe Fall back to system avatar when no member avatar is set 2019-07-26 11:08:06 +02:00
Ske
b14666aa4f Store the ID of the trigger message in the database 2019-07-24 13:38:54 +02:00
Ske
11eabe2e3d Update D.NET version and add bulk deletion support 2019-07-21 17:16:04 +02:00
Ske
4fe80749c4 Fix compiler warnings CS1998 and CS4014 2019-07-21 16:43:28 +02:00
Ske
a078fcdf47 Add description as a field at the bottom of member and system embed 2019-07-21 16:29:48 +02:00
Ske
45f468fe21 Properly clean up after proxy operations 2019-07-21 04:27:04 +02:00
Ske
2620d2da9c Collect webhook cache statistics 2019-07-21 04:15:47 +02:00
Ske
bb6378df63 Add more complex process performance tracking 2019-07-21 00:01:02 +02:00
Ske
b85c79c57b Expand logging capability 2019-07-19 02:29:08 +02:00
Ske
52962f1b97 Bounds check proxy tags before extracting inner message 2019-07-19 02:14:46 +02:00
Ske
1577eb7041 Log channel ID rather than name 2019-07-18 17:29:09 +02:00
Ske
961bfe9094 Add basic logging framework 2019-07-18 17:13:42 +02:00
Ske
dfbb5cd2d6 Add fronter info to system card 2019-07-18 14:05:02 +02:00
Ske
1ad01703b0 Add system tag to embed card 2019-07-18 14:01:28 +02:00
Ske
a188c64a0e Clean up unused imports 2019-07-17 13:38:23 +02:00
Ske
f53344cd89 Fix front percent returning negative percentages and ranges 2019-07-17 13:37:43 +02:00
Ske
02b41413b3 Add basic InfluxDB reporter 2019-07-16 23:34:22 +02:00
Ske
2d58705e85 Add basic WIP metrics system 2019-07-16 21:59:06 +02:00
Ske
d5da3bf004 Clamp frontpercent range to actual switch range 2019-07-16 21:18:46 +02:00
Ske
2f90e7c050 Show sender account in message card 2019-07-15 22:11:08 +02:00
Ske
99919ea6ea Add member color to card 2019-07-15 22:06:08 +02:00
Ske
b6ba24d171 Show front percent for switches with no fronter
Closes #113.
2019-07-15 21:51:41 +02:00
Ske
05f7ead62e Correctly order members in front percent embed 2019-07-15 21:41:24 +02:00
Ske
206d403526 Ignore proxying messages with no content and no attachment 2019-07-15 21:37:34 +02:00
Ske
6668cd8d2d Swallow 404 when deleting proxied message 2019-07-15 21:36:12 +02:00
Ske
3a6e061e44 Avoid making an extra request to fetch message body in webhooks 2019-07-15 17:53:01 +02:00
Ske
da307c2dc9 Display member count on system card 2019-07-15 17:36:10 +02:00
Ske
0a8aeebb23 Fix error showing system card with deleted linked account 2019-07-15 00:05:19 +02:00
Ske
382f533dda Pluralize 'fronter' when applicable 2019-07-14 23:49:14 +02:00
Ske
76d757cae1 Show member fronter card on separate lines 2019-07-14 23:18:51 +02:00
Ske
c6d6a728c9 Fix proxying members named 'Clyde' 2019-07-14 21:27:13 +02:00
Ske
ebc311ecc3 Remove message query reaction AND open DB connection when obtaining one 2019-07-14 05:23:27 +02:00
Ske
d829630a35 Fix database connection pool contention (maybe)
Instead of acquiring a connection per service per request, we
acquire connections more often but at a more granular level, meaning
they're also disposed of more quickly instead of staying for a long time
in case of long-running commands or leaks.
2019-07-11 21:25:23 +02:00
Ske
ca56fd419b Fix various issues with proxying and webhook caching 2019-07-10 23:16:17 +02:00
Ske
740ccf6979 Fix sending message attachments 2019-07-10 12:52:02 +02:00
Ske
5bdb229b34 Fix errant dollar sign in log embed 2019-07-10 12:49:09 +02:00
Ske
42147fd9cc Add front percent command 2019-06-30 23:41:01 +02:00
Ske
7eeaea39fe Proxy messages with a mention before tags 2019-06-27 10:38:45 +02:00
Ske
53037f7d52 Add message querying by ? reaction 2019-06-21 14:13:56 +02:00
Ske
93fff14053 Fix skipping proxying of members with no tags 2019-06-21 13:53:19 +02:00
Ske
6e7950722d Fix log channel clearing 2019-06-21 13:52:34 +02:00
Ske
2c3c46002a Add message lookup and log channel setting commands 2019-06-21 13:49:58 +02:00
Ske
5dafc4fbd4 Add front history command 2019-06-15 12:43:35 +02:00
Ske
f4a53ce815 Refactor date/time format constants 2019-06-15 12:33:24 +02:00
Ske
fa5a616716 Add system fronter command 2019-06-15 12:19:44 +02:00
Ske
08afa2543b Add member avatar edit command
This also refactors a large portion of the DI toolchain, since
I discovered that you shouldn't be reusing IDbConnection objects.

Instead, most services and stores are now declared transient, and
the webhook cache has been moved to a database-independent storage
singleton by itself.
2019-05-17 01:23:09 +02:00
Ske
1824bfd6bb bot: fix proxy service testing members with no tags set 2019-05-13 23:12:58 +02:00
Ske
72a2fadff8 bot: add member deletion command 2019-05-13 23:08:44 +02:00
Ske
b42e052fee bot: add member lookup command 2019-05-12 00:44:02 +02:00
Ske
c5d2b7c251 refactor project structure 2019-05-08 00:06:27 +02:00