Bug,Feature or Design: With PHP cURL cookie needs value for proper caching

serpent_driver

Well-Known Member
#1
I have developed a PHP based Multi-Threading crawler for cache warmup that uses cURL. This crawler also supports multiple cache varies. It works like it should and crawls up to 100,000 URLs within 1 hour, but cache varies drives my crazy. All cache varies work if I request URLs in browser with different cookies and they also work if I warmup the cache with cURL requests in shell script. For cache varies I use different and multiple cookies and also different user agents. Everything works fine, but not if I request URLs in PHP cURL requests with this crawler. For debugging I added cache status and cookies in access_log, but everything looked okay. After hours of testing I found out that cookie needs also cookie value in PHP cURL request.

Code:
CURLOPT_COOKIE => "cookie_1=;cookie_2=" // doesn't work

CURLOPT_COOKIE => "cookie_1=abc;cookie_2=def" // that works
So the question is, is this behaviour caused by PHP cURL or LSWS?
 

winkybil

Active Member
#2
Bit of a PHP n00b who’s been searching for an answer on how to create a proxy server to get around CORS issues. Ultimately, I’m trying to access an external API via a JavaScript fetch request and authenticate a login, set a cookie for future requests, etc.







Kodi nox
 
Last edited:

serpent_driver

Well-Known Member
#4
This not an undefined varibale. Cookie name and its value is an expression. To set cookie with cURL without cookie value there is no need to set a value, but "=" must be set if only cookie name is set. This is RFC compliant.

Please don't talk about things you don't understand.
 

serpent_driver

Well-Known Member
#6
NO! This is cURL and there may only be 1 CURLOPT_COOKIE. Otherwise only the last define is used and all others will be ignored.

Again, please don't talk about things you don't understand.
 

AndreyPopov

Well-Known Member
#7
may be syntax must be right


CURLOPT_COOKIEThe contents of the "Cookie: " header to be used in the HTTP request. Note that multiple cookies are separated with a semicolon followed by a space (e.g., "fruit=apple; colour=red")
 
Top