This change primarily increases runtime performance. The tests added together
are speed up by about 50%. Additionally, binary size decreases by about 15%.

LibreSSL can't be used because it doesn't provide a function to get
the public key for an existing private key in its openssl/curve25519.h.

Signed-off-by: Lukas Lihotzki <lukas@lihotzki.de>