What was happening on our end was that requests were not completely being sent to the server. It was establishing the connection but never sending the request. If we looked at tcpdump we noticed that we got an ACK then a FIN. So then it waits 4 seconds and by the time we come back to get our results curl has timed out.

One major problem is that there’s no way to differentiate between data which needs to be sent from data which needs to be received. We want to block for the first but not for the second (unless we’re asking for said data). This isn’t a problem specifically with PHP’s curl. The underlying curl library doesn’t expose this nor do the system calls.

I have been meaning to patch this. I will do that in the next day or two and then add a comment to this post.

Thanks for the feedback…this bug was a pain in the tail!

I’m having a weird problem with it though:

I have a server application which I call several times in a loop process the responses later. The server expects username and password so I have a static wrapper class which sets all the required cURL options and then calls EpiCurl::getInstance followed by addUrl.

This all works fine if I call in a loop, and then immediately process the results. If, however I simulate some processing by adding a sleep(5) or a big loop, then go process the responses the first response is an authentication error from the server.

So: (inside my static class)

static function wrapper(url)
{
$ch = curl_init();
// set lots of cURL options inc…
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);
curl_setopt($ch, CURLOPT_USERPWD, $username . “:” . $password);
// etc
$mc = EpiCurl::getInstance();
return $mc->addCurl($ch);
}

Then: (in my client code)

$cha = array();
$urls = array(…urls…);
foreach ($urls as $url)
{
$cha[] = staticclass::wrapper($url);
}

sleep(5);

foreach ($cha as $ch)
{
print $ch->data;
}

Without the sleep(5) this works fine on each url. With the sleep(5) the first response is an authentication error.

If I put $x = $cha[0]->code after the first loop (before the sleep(5)) then everything works fine.

Any ideas? This has me stumped!

Thanks,
Alan

first, thanks a lot for your work :)

I had some problems with memory overflow, when running in php-cli. I’m not experienced with classes, so please correct me.

My own function uses class EpiCurl to pull X sources. After that I call destruct, which I added to EpiCurl:

function __destruct()
{
// close handles
foreach($this->requests AS $ch){
curl_close($ch);
}

// close multi handle
if(curl_multi_info_read($this->mc)){
curl_multi_close($this->mc);
}

// unset misc
unset(
$this->requests,
$this->responses,
$this->properties,
$this->running,
$this->mc
);

// reset class
self::__construct();
}

regards, Robert

curl_multi_info_read = PHP 5.2+
http://at.php.net/curl_multi_info_read

I also think it would be good practice to close the curl handles after they are removed from multi-CURL as:

“When a single transfer is completed, the easy handle is still left added to the multi stack. You need to first remove the easy handle with url_multi_remove_handle(3) and then close it with curl_easy_cleanup(3), or possibly set new options to it and add it again with curl_multi_add_handle(3) to start another transfer.”
(source http://curl.haxx.se/libcurl/c/libcurl-multi.html)

Re: credits. Feel free to put in my name, but not email please.

Thanks for pointing that out. Here are the changes I made as a result.

…
if($res === CURLM_OK || $res === CURLM_CALL_MULTI_PERFORM)
{
do {
$mrc = curl_multi_exec($this->mc, $active);
} while ($mrc === CURLM_CALL_MULTI_PERFORM);
…

Is that similar to your fix?

Can I put your name/email in the notes for credits for the fix in the source?

$res = curl_multi_add_handle($this->mc, $ch);
if($res == 0)
{
curl_multi_exec($this->mc, $active);

One of the responses that can be returned by curl_multi_handle is ‘-1′. This is described in the CURL error list (http://curl.haxx.se/libcurl/c/libcurl-errors.html) as:

“CURLM_CALL_MULTI_PERFORM (-1)

This is not really an error. It means you should call curl_multi_perform(3) again without doing select() or similar in between. ”

I don’t think the script should fail on this ‘error’ as it is not really an error. Changing the if statement to execute if a -1 is discovered fixes the problem. NB this does not appear to be a problem with all versions of PHP.

You could do it in the same manner as you would if you were calling different urls. You won’t be guaranteed the order in which the requests are processed though.

In the last sample code simply replace yahoo.com, google.com and slooow.com urls with identical urls.