[PATCH] Support bump-ssl-server-first and mimic SSL server certificates

Hello,

    Attached is a long-awaited patch implementing BumpSslServerFirst and
MimicSslServerCert features. I had to compress the patch to minimize
posting size. If these changes are approved, I plan to "bzr merge" them
into trunk to preserve detailed commit messages (there are too many to
quote all here).

Summary: These changes allow Squid working in SslBump mode to peek at
the origin server certificate and mimic peeked server certificate
properties in the generated fake certificate, all prior to establishing
a secure connection with the client:
http://wiki.squid-cache.org/Features/BumpSslServerFirst
http://wiki.squid-cache.org/Features/MimicSslServerCert

The changes are required to bump intercepted SSL connections without
excessive browser warnings. The changes allow to disable bumping of some
intercepted SSL connections, forcing Squid to go into a TCP tunnel mode
for those connections.

The changes also empower end user to examine and either honor or bypass
most origin SSL server certificate errors. Prior to these changes, the
responsibility for ignoring certificate validation errors belonged
exclusively to Squid, necessarily leaving users in the dark if errors
are ignored/bypassed.

Squid can still be configured to emulate old bump-ssl-client-first
behavior. However, a manual revision of ssl_bump options is required
during upgrade because ssl_bump no longer supports an implicit "negate
the last one" rule (and it is risky to let Squid guess what the admin
true intent was or mix old- and new-style rules).

Finally, fake certificate generation has been significantly improved.
The new code guarantees that all identically configured Squids receiving
identical origin server certificates will generate identical fake
certificates, even if those Squid instances are running on different
hosts, at different times, and do not communicate with each other. Such
stable, reproducible certificates are required for distributed,
scalable, or fail-safe Squid deployment.

Overall, the changes are meant to make SslBump more powerful and safer.
The code has been tested in several independent labs.

Specific major changes are highlighted below:

Make bumping algorithm selectable using ACLs. Even though
bump-server-first is an overall better method, bumping the client first
is useful for backward compatibility and possibly for serving internal
Squid objects (such as icons inside Squid error pages). The following
example bumps special and most other requests only, using the old
bump-client-first approach for the special requests only:

  ssl_bump client-first specialOnes
  ssl_bump server-first mostOthers
  ssl_bump none all

Added sslproxy_cert_adapt squid.conf option to overwrite default
mimicking behavior when generating SSL certificates. See
squid.conf.documented.

Added sslproxy_cert_sign squid.conf option to control how generated SSL
certificates are signed. See squid.conf.documented.

Added ssl::certHasExpired, ssl::certNotYetValid,
ssl::certDomainMismatch, ssl::certUntrusted, and ssl::certSelfSign
predefined ACLs to squid.conf.

Do not require http[s]_port's key option to be set if cert option is
given. The fixed behavior for bumped connections now matches squid.conf
docs.

Generate stable fake certificates by using signing and true certificate
hashes as the serial number and by using the configured CA private key
for all fake certificates.

Use minimal, trusted certificate for serving SSL errors to the client
instead of trying to mimic the broken true certificate (which results in
double error for the user: browser error dialog plus Squid error page).

To mimic "untrusted" true certificates, generate an untrusted CA
certificate from the configured trusted CA certificate. This both
reduces configuration effort (compared to a configuration option) and
results in identical untrusted fake certificates given identical Squid
configurations.

Intelligent handling of CONNECT denials: Do not connect to origin
servers unless CONNECT is successfully authenticated. Delay errors.

Provide '%I' error page formatting code with enough information to avoid
displaying '[unknown]' on SQUID_X509_V_ERR_DOMAIN_MISMATCH errors.

Set logged status code (%<Hs) to 200 when establishing a bumped tunnel.

Improved error detailing and logging: Forget most retried errors. During
SslBump errors, the error details are now logged with both the initial
CONNECT transaction and the first tunneled HTTP transaction. Do not
report system errors as custom Squid errors. Do not report system errors
that did not necessarily happen during the transaction being logged.

Check SSL server certificate when reconnecting to the origin server for
bumped requests. Despite pinning, Squid maintains two separate
connections and the server may disconnect while the client is still
sending requests. To minimize deployment problems, we reconnect to the
origin server but check that its certificate (which we mimicked for the
client) has not changed much.

Forward bumped server connection-close signal to the bumped client to
improve the "dumb tunnel" appearance of the bumped SSL tunnel.

Allow bumping of CONNECT requests without allow-direct set on http_port.
Previously, that flag was required to allow bumped requests to go direct
because they were (and, sometimes, still are) considered "accelerated".

Send SNI information to the server when server-first bumping a non-IP
CONNECT request.

Better helper-to-Squid buffer size management to support large
certificates.

Bypass rare OpenSSL certificate serialization failures when composing an
ssl_crtd request by generating the certificate in the Squid process.

When generating certificate CN names, strip [] surrounding host names,
assuming they are for IPv6 addresses. Bracketed CNs confuse browsers.

Disable persistent connections after client-side-detected errors. They
cause "abandoning such and such connection" warnings, stuck
ConnStateData jobs, and other problems.

HttpRequest::SetHost() must invalidate HttpRequest::canonical "cache".

Synced with trunk (trunk r12181, v3.2.0.17+)

Thank you,

Alex.