Set parallel-max-host if curl>=8.16.0

 If an user provides a multiple URLs, we don't want to open too many
 connections to the same host in parallel due to the risk of getting
 throttled.

 wcurl will use the newly added option --parallel-max-host if available,
 using 5 as the default value per host.

 This contains an update to the docs to show how an user can remove the
 limit.

Author: samueloph

README.md

@@ -63,6 +63,7 @@ should be using curl directly if your use case is not covered.
 * By default, **wcurl** does:
   * Percent-encode whitespaces in URLs;
   * Download multiple URLs in parallel if the installed curl's version is >= 7.66.0 (`--parallel`);
+  * Use a total number of 5 parallel connections to the same protocol + hostname + port number target if the installed curl's version is >= 8.16.0 (`--parallel-max-host`);
   * Follow redirects;
   * Automatically choose a filename as output;
   * Avoid overwriting files if the installed curl's version is >= 7.83.0 (`--no-clobber`);
@@ -139,6 +140,12 @@ then performs the parsing. May be specified more than once.
   wcurl --curl-options="--continue-at -" example.com/filename.txt
   ```
 
+* Download multiple files without a limit in number of concurrent connections per host, the default is 5:
+
+  ```sh
+  wcurl --curl-options="--parallel-max-host 0" example.com/filename1.txt example.com/filename2.txt
+  ```
+
 # Running the testsuite
 
 If you would like to run the tests, you first need to install the

wcurl

@@ -181,18 +181,23 @@ exec_curl()
 {
     CMD="curl "
 
-    # Store version to check if it supports --no-clobber and --parallel.
+    # Store version to check if it supports --no-clobber, --parallel and --parallel-max-host.
     curl_version=$($CMD --version | cut -f2 -d' ' | head -n1)
     curl_version_major=$(echo "$curl_version" | cut -f1 -d.)
     curl_version_minor=$(echo "$curl_version" | cut -f2 -d.)
 
     CURL_HAS_NO_CLOBBER=""
     CURL_HAS_PARALLEL=""
+    CURL_HAS_PARALLEL_MAX_HOST=""
     # --no-clobber is only supported since 7.83.0.
     # --parallel is only supported since 7.66.0.
+    # --parallel-max-host is only supported since 8.16.0.
     if [ "${curl_version_major}" -ge 8 ]; then
         CURL_HAS_NO_CLOBBER="--no-clobber"
         CURL_HAS_PARALLEL="--parallel"
+        if [ "${curl_version_minor}" -ge 16 ]; then
+            CURL_HAS_PARALLEL_MAX_HOST="--parallel-max-host 5"
+        fi
     elif [ "${curl_version_major}" -eq 7 ]; then
         if [ "${curl_version_minor}" -ge 83 ]; then
             CURL_HAS_NO_CLOBBER="--no-clobber"
@@ -202,13 +207,13 @@ exec_curl()
         fi
     fi
 
-    # Detecting whether we need --parallel.  It's easier to rely on
+    # Detecting whether we need --parallel. It's easier to rely on
     # the shell's argument parsing.
     # shellcheck disable=SC2086
     set -- $URLS
 
     if [ "$#" -gt 1 ]; then
-        CURL_PARALLEL="$CURL_HAS_PARALLEL"
+        CURL_PARALLEL="$CURL_HAS_PARALLEL ${CURL_HAS_PARALLEL_MAX_HOST}"
     else
         CURL_PARALLEL=""
     fi

wcurl.md

@@ -45,6 +45,9 @@ By default, **wcurl** does:
 ## * Download multiple URLs in parallel
     if the installed curl's version is \>= 7.66.0 (--parallel);
 
+## * Use a total number of 5 parallel connections to the same protocol + hostname + port number target
+    if the installed curl's version is \>= 8.16.0 (--parallel-max-host);
+
 ## * Follow redirects;
 
 ## * Automatically choose a filename as output;
@@ -129,6 +132,10 @@ be the last one in the list):
 
 **wcurl --curl-options="--continue-at -" example.com/filename.txt**
 
+Download multiple files without a limit in number of concurrent connections per host, the default is 5:
+
+**wcurl --curl-options="--parallel-max-host 0" example.com/filename1.txt example.com/filename2.txt**
+
 # AUTHORS
 
     Samuel Henrique \<[email protected]\>