Commit Graph

22 Commits

Author SHA1 Message Date
io 290bb33538 use external pleroma.py 2023-01-11 06:30:47 +00:00
Kay Faraday f7cc00e443 fetch_posts.py: remove lots of dead code 2022-12-28 23:07:02 +00:00
Kay Faraday f58d85635a fetch_posts.py: support Wordpress ActivityPub plugin 2022-12-28 23:06:23 +00:00
Kay Faraday 5e50bccd53 deduplicate user-agent setting code 2022-06-19 02:02:32 +00:00
Kay Faraday b73526a895 simplify account fetch by removing the WebFinger step
Now we fetch the user's profile URL from our home server, skipping the need to WebFinger them
and also supporting instances that don't support http:// connections.
2022-06-15 19:53:43 +00:00
io 94949289f0 normalize Pleroman posts without cws (fix #3) 2021-09-27 11:17:56 +00:00
io 4492f6685c remove repeated call to extract_post_content (#2)
fortunately, extract_post_content returns the same thing when called repeatedly,
so this wasn't a big deal anyway
2021-09-27 11:17:56 +00:00
ioistired 3d393c7577
Merge pull request #2 from TinfoilSubmarine/create-posts-on-first-run
Commit on DB object creation to ensure table posts is created
2021-09-27 05:42:46 -05:00
io 2bea936cf2 fix fetching from masto instances 2021-09-21 13:58:20 +00:00
Joel Beckmeyer 6f0a389050 create table posts if it does not exist 2021-09-17 14:07:32 -04:00
io da961a0e35 add TODO comment 2021-09-17 06:37:39 +00:00
io 788f8550c7 handle rate limits 2021-09-17 06:35:54 +00:00
io b906abe2b1 add basic migration support 2021-09-17 06:35:35 +00:00
io 191214dbd6 save some memory by using account handles instead of objects 2021-08-19 10:40:57 +00:00
io 97a2f5de00 a few changes
- better error reporting
- fix the program hanging at the end if any of the accounts failed to fetch
- remove trailing semicolon from user agent string
2021-08-13 09:40:20 +00:00
io 9b60540d07 move db_path default to load_config 2021-07-26 06:47:43 +00:00
io a3348602c1 fetch_posts: add KeyboardInterrupt handling 2021-07-26 06:17:01 +00:00
io 4ba3bb1662 fetch_posts.py: use argparse 2021-07-26 06:04:32 +00:00
io b8e727cfd4 fetch_posts.py: remove unused import 2021-07-26 06:04:14 +00:00
io 330fdc2809 rewrite fetch_posts.py from scratch
now it should be properly async by fetching posts in a separate task and sending them across
a queue to a task that inserts to the DB
2021-07-26 04:59:04 +00:00
io 944e2fc3a5 use json5 instead lol
toml sucks add null when
2021-06-18 10:38:58 +00:00
io 5f9959fff8 rename main.py to fetch_posts.py 2021-06-16 04:26:22 +00:00
Renamed from main.py (Browse further)