[Resolved] LiteSpeed not returning Content-Encoding header for Range requests, breaks gzip

Status
Not open for further replies.
S

Sean Kinsey

New Member
#1
#1
When combining `Accept-Encoding: deflate, gzip` and `Range: bytes=0-524281` as request headers, the response, both gzipped and limited to the range, is returned with a missing `Content-Encoding` header.
This causes Curl and other clients to fail to decode the response.

Code: 
oyvind-mbp:~ oyvind$ curl -v -A "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" -H "Range: bytes=0-524281" --compressed "http://modernpracticality.com/"
* Adding handle: conn: 0x7fe42c003a00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fe42c003a00) send_pipe: 1, recv_pipe: 0
* About to connect() to modernpracticality.com port 80 (#0)
*   Trying 173.248.188.234...
* Connected to modernpracticality.com (173.248.188.234) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
> Host: modernpracticality.com
> Accept: */*
> Accept-Encoding: deflate, gzip
> Range: bytes=0-524281
>
< HTTP/1.1 206 Partial Content
< Date: Fri, 01 Aug 2014 04:50:32 GMT
* Server LiteSpeed is not blacklisted
< Server: LiteSpeed
< Accept-Ranges: bytes
< Connection: Keep-Alive
< Keep-Alive: timeout=5, max=100
< Last-Modified: Fri, 01 Aug 2014 04:48:10 GMT
< Content-Type: text/html
< X-Pingback: http://modernpracticality.com/xmlrpc.php
< Vary: Accept-Encoding, Cookie
< Content-Range: bytes 0-26570/26571
< Content-Length: 26571
We have seen this with multiple LiteSpeed powered domains, and so believe this is related to LiteSpeed, or how it interacts with other proxy components.

( referenced from https://developers.facebook.com/bugs/335400146618351/ )
 
Last edited by a moderator:
M

mistwang

LiteSpeed Staff
#2
#2
For Range request handled by LiteSpeed Enterprise Web Server itself, gzip compression is turned off, in other words, LiteSpeed does not apply gzip compressed to range requests.

If it was compressed, it must be done by something else.

When I try your test case, the page is served by PHP directly, without responding to the range request.
 
S

Sean Kinsey

New Member
#3
#3
Hey @mistwang, although you might be right about LiteSpeed being self-consistent, this is an issue that we are only seeing with LiteSpeed servers. Attached is another sample:

Code: 
$ curl -v --compressed -H 'Connection: close' -H 'Range: bytes=0-524288' -A "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" "http://www.republica.com.uy/" | head
* Hostname was NOT found in DNS cache
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 72.52.244.255...
* Connected to www.republica.com.uy (72.52.244.255) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
> Host: www.republica.com.uy
> Accept: */*
> Accept-Encoding: deflate, gzip
> Connection: close
> Range: bytes=0-524288
>
< HTTP/1.1 206 Partial Content
< Date: Tue, 18 Nov 2014 18:10:02 GMT
* Server LiteSpeed is not blacklisted
< Server: LiteSpeed
< Accept-Ranges: bytes
< Connection: close
< Last-Modified: Tue, 18 Nov 2014 18:09:54 GMT
< Content-Type: text/html; charset=UTF-8
< X-Pingback: http://www.republica.com.uy/xmlrpc.php
< Vary: Accept-Encoding, Cookie
< Pragma: public
< Cache-Control: public
< Content-Range: bytes 0-22802/22803
< Content-Length: 22803
<
{ [data not shown]
�Ks#G�.���!�Hv!�"Y�G�:,�%���bY��th����D&���j��Y�屻��ؽf��5�B�1�z3f�2�����H    `���f��Ȍ�G�����?���f�����=q���mQ���?/n��;G;�/_��j�"�B�Gn�������Qh�qg�\>;;+�-���Y>z[>G[UT6�V��Yrb���y�onT��J:��=��(T�F!jal'�p��/�V��B�I�k˒�KI�|ֱ�d���T[EeOg��k��b�!O�\��)�r�;Ǐ��b���V.SY_�q�!mU��%_Ž�\j��Ԡ��R���7Xe;��lȶ�u7<�mH7|��Fz^[�T���ʜ@�sQ��T�R*�q��6�bu������:��'�4���3��ucI��_����k�>X�[���P�|֍�i��,�T�0
 
Last edited by a moderator:
M

mistwang

LiteSpeed Staff
#4
#4
Please try the latest 4.2.19 build, we fixed a gzip related issue yesterday.
If it does not fix it, we may need to login and take a look at it.
 
S

Sean Kinsey

New Member
#5
#5
Hi @mistwang, I'm actually reporting this on behalf of Facebook, so won't be able to upgrade any implementations.
I will refer bug reporters to this thread though.

Would you be able to have someone verify if the bug could present itself as described in this thread?
 
M

mistwang

LiteSpeed Staff
#6
#6
Sure, we will work with our client to figure it out.
I probably know what is problem. Need to find a solution.

Will facebook side takes a "200" response instead of "206 Partial Content" response contains the whole file?
 
M

mistwang

LiteSpeed Staff
#8
#8
This issue should have been addressed in our latest 4.2.19 build. Any user reporting this issue should upgrade to 4.2.19 with
Code: 
/usr/local/lsws/admin/misc/lsup.sh -f -v 4.2.19
 
Last edited by a moderator:
B

brightestspark

Member
#9
#9
My host (VentraIP) is running Litespeed Server API 6.7. I have several wordpress sites on this server. They all have the Wordfence plugin. It has a gzip caching system called Falcon Engine. When that is enabled, I get garbled output on Facebook scrapes. Attached is what Facebook sees after a fresh scrape of one of my websites. Compress content is disabled in cPanel. I have no other caching/gzip functions. Wordfence keep telling me that my content is getting gzipped twice, which it is not.
Refer https://wordpress.org/support/topic...ine-gzip-on-litespeed?replies=11#post-6272195
 

Attachments

B

brightestspark

Member
#11
#11
Update - I just checked with VentraIP and they said they upgraded to 4.2.19 two weeks ago. The problem still remains with my websites. I have kept Wordfence Falcon Engine (gzip caching) enabled on this site mullewashow.com.au if you want to check via https://developers.facebook.com/tools/debug/og/object/
 
Last edited by a moderator:
N

NiteWave

Administrator
#12
#12
please ask them to do a force-reinstall to ensure the latest 4.2.19
on command line:
Code: 
/usr/local/lsws/admin/misc/lsup.sh -f -v 4.2.19
 
Last edited by a moderator:
B

brightestspark

Member
#13
#13
This is what VentraIP says:

"The strange thing here is - the Litespeed version referenced in those posts (4.2.19) is the same as what's already installed. (Our WHM reports 4.2.19 as well)"
 
B

brightestspark

Member
#15
#15
Apologies if I am repeating myself, this is VentraIPs reply:

Could you please let them know that we are already running the latest build of LiteSpeed/4.2.19 Enterprise.

If they simply request a force install again then please let us know.
 
N

NiteWave

Administrator
#16
#16
yes, just a simple force install
to compare:
Code: 
ls -l /usr/local/lsws/bin/lshttpd.4.2.19
to know when they installed lsws 4.2.19 last time.

then simply do a force re-install
Code: 
/usr/local/lsws/admin/misc/lsup.sh -f -v 4.2.19
or through lsws web admin
Actions->Version Manager->4.2.19:Force Reinstal

then your issue should be resolved.
 
Last edited by a moderator:
Status
Not open for further replies.
Top