The issue was that openresty debian did not ship with the http slice module which allowed byte range requests. Since we have 200mb+ files, that means our local S3 had to serve 200MB requests, send them to the middle servers, and have those servers cache and send byte ranges out.
I assumed this was "a problem" but I didn't really appreciate just how big of a problem it was. When we started hitting 3000+ users again I realized there was a serious I/O issue and it was causing database errors. I gave the database ionice priority and that helped a lot, which surprised me because our storage is all SSDs. The SSDs were peaking at over 750Mbps of read. I assumed this was just the natural usage of the S3 server, since the IO was on that and not the webserver. At some point it clicked that all the problems were unified by a common theme of large files not sending as byte ranges and yelled at the vendors to ship with slice. It cut IO read down by more than 1/10th.
In retrospect I think that any person opening any page with videos caused the S3 to load and send the entire video just to generate the first bit of the byte range for a preview image, i.e. every video tag had a full download impact on S3.