Wednesday, October 11, 2017

Multipart ranges in Nginx reverse proxy

I am trying to setup Nginx as a reverse proxy. The upstream server is serving some media files.




Because of the large amount of requests for these files and also since these files are not going to change for at least a couple of weeks, I'm caching the upstream responses in Nginx and serving all subsequent requests from the cache.



proxy_cache_path /home/bandc/content levels=1:2 keys_zone=content_cache:10m max_size=10g inactive=15d use_temp_path=off;

upstream mycdn {
server myserver.dev;
}

location / {

proxy_cache content_cache;
proxy_pass http://mycdn;
proxy_cache_methods GET HEAD;
proxy_cache_valid 200 302 7d;
proxy_cache_valid 404 10m;
add_header x-cache $upstream_cache_status;
....
}



The clients can send Range requests for large media files. The upstream server does support range requests.



The problem is if I send a request with multiple byte ranges before sending any GET or single Range request (i.e the response hasn't been cached before this multiple byte ranges request), Nginx delivers the whole file with 200 OK instead of the requested ranges with 206 Partial Content. But once the content has been cached, all multipart range requests work flowlessly.



I looked around a bit and found this blog post:




How Does NGINX Handle Byte Range Requests?
If the file is up‑to‑date in the cache, then NGINX honors a byte range request and serves only the specified bytes of the item to the client. If the file is not cached, or if it’s stale, NGINX downloads the entire file from the origin server. If the request is for a single byte range, NGINX sends that range to the client as soon as it is encountered in the download stream. If the request specifies multiple byte ranges within the same file, NGINX delivers the entire file to the client when the download completes.





Is there any way to ensure that if the file is not cached yet, multipart range should be served from upstream alone (without caching) and eventually from local cache after Nginx caches it when GET or a Range request with a single byte range is performed?

No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...