2 commits

Author	SHA1	Message	Date
Zed	d7352b0552	Explicitly allow Twitterbot to generate previews	2022-09-14 09:08:40 +02:00
minus	b6d1947747	Block search engines via robots.txt (#631) ... Prevents instances from being rate limited due to being senselessly crawled by search engines. Since there is no reason to index Nitter instances, simply block all robots. Notably, this does *not* affect link previews (e.g. in various chat software).	2022-09-14 09:08:40 +02:00