Google only specifies what happens when the connection is not used because the HTTP specs already have information how connections should stay open when they are used.
If a connection is used, it is up to the server to keep the connection open so that it can be used again. This is known as "HTTP Keep Alive".
For example my Apache server comes with this default (Ubuntu/Debian) configuration:
This allows connections to stay open for 5 seconds after they are used waiting for a new request. The connection can be used for 100 back to back requests before the server kills the connection.