[PHP] Collecting URL list and DL

Aug 23, 2020 PHP curl

For the time being, as a memorandum

We will create a list of URLs that can be DLed together. For example, you can write a URL in a text file like the one below and DL them together.

https://.....
https://.....
https://.....
https://.....

Create #DL save directory The name of the directory is the same as the name of the URL list file. Takes the URL list file path as a command line argument and extracts the name from it.

Below is the code to create if there is no save directory

$dirName = basename($argv[1], ".txt");
$dir = FILE .'src/' .$dirName;
if (!file_exists($dir)) {
    mkdir($dir);
}

##basename Makes it easy to extract the file name from the path. The first argument is the path, and if you specify the extension in the second argument, you can extract only the file name.

#Read URL list text with command line arguments Considering that the URL will be huge, read one line at a time and make the URL list file an array

Below is the template

$file = fopen($argv[1], "r");
if ($file) {
    while ($line = fgets($file)) {
       //processing
    }
}
fclose($file);

Reference: https://webkaru.net/php/function-fgets/

#download I turned around with for, but devised the beginning of the loop

for ($i = $argv[2] ?? 0; $i <count($array); $i++) {
//processing
}

The initialization of $i is done like $i = $argv[2] ?? 0;. This is to assign $argv[2] if it is, and to substitute 0 otherwise. This allows you to specify where you want the DL to start.

##curl I only write the basics, but I will post it for the time being.

    $options = array(
        CURLOPT_URL => $url,
        CURLOPT_RETURNTRANSFER => true,
    );

    $ch = curl_init();

    //option
    curl_setopt_array($ch, $options);

    $html = curl_exec($ch);
    file_put_contents($filepath, $html);
    curl_close($ch);

It seems that saving did not work unless CURLOPT_RETURNTRANSFER => true was specified…

#Execute

$php dl.php URL list file path [optional]