lib: rsa: Set conventional salt length RSA-PSS parameter

RFC 3447 says that Typical salt length are either 0 or the length
of the output of the digest algorithm, RFC 4055 also recommends
hash value length as the salt length. Moreover, By convention,
most of the signing infrastructures/libraries use the length of
the digest algorithm (such as google cloud kms:
                      https://cloud.google.com/kms/docs/algorithms).

If the salt-length parameter is not set, openssl default to the
maximum allowed value, which is a openssl 'specificity', so this
works well for local signing, but restricts compatibility with
other engines (e.g pkcs11/libkmsp11):

```
returning 0x71 from C_SignInit due to status INVALID_ARGUMENT:
    at rsassa_pss.cc:53: expected salt length for key XX is 32,
    but 478 was supplied in the parameters
Could not obtain signature: error:41000070:PKCS#11 module::Mechanism invalid
```

To improve compatibility, we set the default RSA-PSS salt-length
value to the conventional one. A further improvement could consist
in making it configurable as signature FIT node attribute.

rfc3447: https://datatracker.ietf.org/doc/html/rfc3447
rfc4055: https://datatracker.ietf.org/doc/html/rfc4055

Signed-off-by: Loic Poulain <[email protected]>

0 files changed, 0 insertions, 0 deletions