19 Commits

Author	SHA1	Message	Date
Kay Faraday	5e50bccd53	deduplicate user-agent setting code	2022-06-19 02:02:32 +00:00
Kay Faraday	b73526a895	simplify account fetch by removing the WebFinger step ... Now we fetch the user's profile URL from our home server, skipping the need to WebFinger them and also supporting instances that don't support http:// connections.	2022-06-15 19:53:43 +00:00
io	94949289f0	normalize Pleroman posts without cws (fix #3)	2021-09-27 11:17:56 +00:00
io	4492f6685c	remove repeated call to extract_post_content (#2) ... fortunately, extract_post_content returns the same thing when called repeatedly, so this wasn't a big deal anyway	2021-09-27 11:17:56 +00:00
ioistired	3d393c7577	Merge pull request #2 from TinfoilSubmarine/create-posts-on-first-run ... Commit on DB object creation to ensure table posts is created	2021-09-27 05:42:46 -05:00
io	2bea936cf2	fix fetching from masto instances	2021-09-21 13:58:20 +00:00
Joel Beckmeyer	6f0a389050	create table posts if it does not exist	2021-09-17 14:07:32 -04:00
io	da961a0e35	add TODO comment	2021-09-17 06:37:39 +00:00
io	788f8550c7	handle rate limits	2021-09-17 06:35:54 +00:00
io	b906abe2b1	add basic migration support	2021-09-17 06:35:35 +00:00
io	191214dbd6	save some memory by using account handles instead of objects	2021-08-19 10:40:57 +00:00
io	97a2f5de00	a few changes ... - better error reporting - fix the program hanging at the end if any of the accounts failed to fetch - remove trailing semicolon from user agent string	2021-08-13 09:40:20 +00:00
io	9b60540d07	move db_path default to load_config	2021-07-26 06:47:43 +00:00
io	a3348602c1	fetch_posts: add KeyboardInterrupt handling	2021-07-26 06:17:01 +00:00
io	4ba3bb1662	fetch_posts.py: use argparse	2021-07-26 06:04:32 +00:00
io	b8e727cfd4	fetch_posts.py: remove unused import	2021-07-26 06:04:14 +00:00
io	330fdc2809	rewrite fetch_posts.py from scratch ... now it should be properly async by fetching posts in a separate task and sending them across a queue to a task that inserts to the DB	2021-07-26 04:59:04 +00:00
io	944e2fc3a5	use json5 instead lol ... toml sucks add null when	2021-06-18 10:38:58 +00:00
io	5f9959fff8	rename main.py to fetch_posts.py	2021-06-16 04:26:22 +00:00

Renamed from main.py (Browse further)