Commit 5d1b66c3 authored by manuroe's avatar manuroe
Browse files

Merge remote-tracking branch 'origin/master' into olmkit

parents 62f52806 f6c05be8
......@@ -2,6 +2,7 @@
/CHANGELOG.html
/docs/megolm.html
/docs/olm.html
/docs/signing.html
/olm-*.tgz
/README.html
/tracing/README.html
......
Changes in `2.0.0 <http://matrix.org/git/olm/commit/?h=2.0.0>`_
===============================================================
This release includes the following changes since 1.3.0:
* Fix a buffer bounds check when decoding group messages.
* Update ``olm_group_decrypt`` to return the ratchet index for decrypted
messages.
* Fix ``olm_pickle_account``, ``olm_pickle_session``,
``olm_pickle_inbound_group_session`` and
``olm_pickle_outbound_group_session`` to correctly return the length of the
pickled object.
* Add a `specification <./docs/megolm.rst>`_ of the Megolm ratchet, and add
some information on mitigating unknown key-share attacks to the `Olm
specification <./docs/olm.rst>`_.
* Add an ``install-headers`` target to the Makefile (and run it when installing
the library). (Credit to Emmanuel Gil Peyrot).
Changes in `1.3.0 <http://matrix.org/git/olm/commit/?h=1.3.0>`_
===============================================================
The release updates the group session identifier to avoid collisions.
The group sessions are now identified by their ed25519 public key.
This release updates the group session identifier to avoid collisions.
Group sessions are now identified by their ed25519 public key.
These changes alter the pickle format of outbound group sessions, attempting
to unpickle an outbound group session created with a previous version of olm
......
#!/usr/bin/make -f
MAJOR := 1
MINOR := 3
MAJOR := 2
MINOR := 0
PATCH := 0
VERSION := $(MAJOR).$(MINOR).$(PATCH)
PREFIX ?= /usr/local
......@@ -46,6 +46,7 @@ JS_POST := javascript/olm_outbound_group_session.js \
DOCS := tracing/README.html \
docs/megolm.html \
docs/olm.html \
docs/signing.html \
README.html \
CHANGELOG.html
......
......@@ -5,8 +5,12 @@ An implementation of the Double Ratchet cryptographic ratchet described by
https://github.com/trevp/double_ratchet/wiki, written in C and C++11 and
exposed as a C API.
The specification of the Olm ratchet can be found in docs/olm.rst or
https://matrix.org/docs/spec/olm.html
The specification of the Olm ratchet can be found in ``docs/olm.rst`` or
https://matrix.org/docs/spec/olm.html.
This library also includes an implementation of the Megolm cryptographic
ratchet, as specified in ``docs/megolm.rst`` or
https://matrix.org/docs/spec/megolm.html.
Building
--------
......
......@@ -27,7 +27,7 @@ The Megolm ratchet is intended for encrypted messaging applications where there
may be a large number of recipients of each message, thus precluding the use of
peer-to-peer encryption systems such as `Olm`_.
It also allows a receipient to decrypt received messages multiple times. For
It also allows a recipient to decrypt received messages multiple times. For
instance, in client/server applications, a copy of the ciphertext can be stored
on the (untrusted) server, while the client need only store the session keys.
......@@ -143,7 +143,7 @@ copy of the counter, ratchet, and public key.
Message encryption
~~~~~~~~~~~~~~~~~~
This version of Megolm uses AES-256_ in CBC_ mode with `PCKS#7`_ padding and
This version of Megolm uses AES-256_ in CBC_ mode with `PKCS#7`_ padding and
HMAC-SHA-256_ (truncated to 64 bits). The 256 bit AES key, 256 bit HMAC key,
and 128 bit AES IV are derived from the megolm ratchet :math:`R_i`:
......@@ -199,9 +199,9 @@ session.
In order to maintain the ability to decrypt conversation history, inbound
sessions should store a copy of their earliest known ratchet value (unless they
explicitly want to drop the ability to decrypt that history). They may also
choose to cache calculated ratchet values, but the decision of which ratchet
states to cache is left to the application.
explicitly want to drop the ability to decrypt that history - see `Partial
Forward Secrecy`_\ ). They may also choose to cache calculated ratchet values,
but the decision of which ratchet states to cache is left to the application.
Data exchange formats
---------------------
......@@ -269,7 +269,79 @@ protocol). The MAC protects all of the bytes preceding the MAC.
The length of the signature is determined by the signing algorithm being used
(64 bytes in this version of the protocol). The signature covers all of the
bytes preceding the signaure.
bytes preceding the signature.
Limitations
-----------
Message Replays
---------------
A message can be decrypted successfully multiple times. This means that an
attacker can re-send a copy of an old message, and the recipient will treat it
as a new message.
To mitigate this it is recommended that applications track the ratchet indices
they have received and that they reject messages with a ratchet index that
they have already decrypted.
Lack of Transcript Consistency
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In a group conversation, there is no guarantee that all recipients have
received the same messages. For example, if Alice is in a conversation with Bob
and Charlie, she could send different messages to Bob and Charlie, or could
send some messages to Bob but not Charlie, or vice versa.
Solving this is, in general, a hard problem, particularly in a protocol which
does not guarantee in-order message delivery. For now it remains the subject of
future research.
Lack of Backward Secrecy
~~~~~~~~~~~~~~~~~~~~~~~~
Once the key to a Megolm session is compromised, the attacker can decrypt any
future messages sent via that session.
In order to mitigate this, the application should ensure that Megolm sessions
are not used indefinitely. Instead it should periodically start a new session,
with new keys shared over a secure channel.
.. TODO: Can we recommend sensible lifetimes for Megolm sessions? Probably
depends how paranoid we're feeling, but some guidelines might be useful.
Partial Forward Secrecy
~~~~~~~~~~~~~~~~~~~~~~~
Each recipient maintains a record of the ratchet value which allows them to
decrypt any messages sent in the session after the corresponding point in the
conversation. If this value is compromised, an attacker can similarly decrypt
those past messages.
To mitigate this issue, the application should offer the user the option to
discard historical conversations, by winding forward any stored ratchet values,
or discarding sessions altogether.
Dependency on secure channel for key exchange
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The design of the Megolm ratchet relies on the availability of a secure
peer-to-peer channel for the exchange of session keys. Any vulnerabilities in
the underlying channel are likely to be amplified when applied to Megolm
session setup.
For example, if the peer-to-peer channel is vulnerable to an unknown key-share
attack, the entire Megolm session become similarly vulnerable. For example:
Alice starts a group chat with Eve, and shares the session keys with Eve. Eve
uses the unknown key-share attack to forward the session keys to Bob, who
believes Alice is starting the session with him. Eve then forwards messages
from the Megolm session to Bob, who again believes they are coming from
Alice. Provided the peer-to-peer channel is not vulnerable to this attack, Bob
will realise that the key-sharing message was forwarded by Eve, and can treat
the Megolm session as a forgery.
A second example: if the peer-to-peer channel is vulnerable to a replay
attack, this can be extended to entire Megolm sessions.
License
-------
......@@ -285,6 +357,6 @@ Version 2.0 <http://www.apache.org/licenses/LICENSE-2.0>`_.
.. _`SHA-256`: https://tools.ietf.org/html/rfc6234
.. _`AES-256`: http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf
.. _`CBC`: http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf
.. _`PCKS#7`: https://tools.ietf.org/html/rfc2315
.. _`PKCS#7`: https://tools.ietf.org/html/rfc2315
.. _`Olm`: ./olm.html
.. _`Protocol Buffers encoding`: https://developers.google.com/protocol-buffers/docs/encoding
......@@ -30,7 +30,7 @@ Initial setup
~~~~~~~~~~~~~
The setup takes four Curve25519_ inputs: Identity keys for Alice and Bob,
:math:`I_A` and :math:`I_B`, and ephemeral keys for Alice and Bob,
:math:`I_A` and :math:`I_B`, and one-time keys for Alice and Bob,
:math:`E_A` and :math:`E_B`. A shared secret, :math:`S`, is generated using
`Triple Diffie-Hellman`_. The initial 256 bit root key, :math:`R_0`, and 256
bit chain key, :math:`C_{0,0}`, are derived from the shared secret using an
......@@ -279,7 +279,7 @@ Olm Authenticated Encryption
Version 1
~~~~~~~~~
Version 1 of Olm uses AES-256_ in CBC_ mode with `PCKS#7`_ padding for
Version 1 of Olm uses AES-256_ in CBC_ mode with `PKCS#7`_ padding for
encryption and HMAC-SHA-256_ (truncated to 64 bits) for authentication. The
256 bit AES key, 256 bit HMAC key, and 128 bit AES IV are derived from the
message key using HKDF-SHA-256_ using the default salt and an info of
......@@ -298,6 +298,37 @@ and the IV :math:`AES\_IV_{i,j}` to give the cipher-text, :math:`X_{i,j}`.
Then the entire message (including the Version Byte and all Payload Bytes) are
passed through HMAC-SHA-256. The first 8 bytes of the MAC are appended to the message.
Message authentication concerns
-------------------------------
To avoid unknown key-share attacks, the application must include identifying
data for the sending and receiving user in the plain-text of (at least) the
pre-key messages. Such data could be a user ID, a telephone number;
alternatively it could be the public part of a keypair which the relevant user
has proven ownership of.
.. admonition:: Example attacks
1. Alice publishes her public Curve25519 identity key, :math:`I_A`. Eve
publishes the same identity key, claiming it as her own. Bob downloads
Eve's keys, and associates :math:`I_A` with Eve. Alice sends a message to
Bob; Eve intercepts it before forwarding it to Bob. Bob believes the
message came from Eve rather than Alice.
This is prevented if Alice includes her user ID in the plain-text of the
pre-key message, so that Bob can see that the message was sent by Alice
originally.
2. Bob publishes his public Curve25519 identity key, :math:`I_B`. Eve
publishes the same identity key, claiming it as her own. Alice downloads
Eve's keys, and associates :math:`I_B` with Eve. Alice sends a message to
Eve; Eve cannot decrypt it, but forwards it to Bob. Bob believes the
Alice sent the message to him, wheras Alice intended it to go to Eve.
This is prevented by Alice including the user ID of the intended recpient
(Eve) in the plain-text of the pre-key message. Bob can now tell that the
message was meant for Eve rather than him.
IPR
---
......@@ -323,4 +354,4 @@ an entirely new implementation written by the Matrix.org team.
.. _`SHA-256`: https://tools.ietf.org/html/rfc6234
.. _`AES-256`: http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf
.. _`CBC`: http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf
.. _`PCKS#7`: https://tools.ietf.org/html/rfc2315
.. _`PKCS#7`: https://tools.ietf.org/html/rfc2315
.. Copyright 2016 OpenMarket Ltd
..
.. Licensed under the Apache License, Version 2.0 (the "License");
.. you may not use this file except in compliance with the License.
.. You may obtain a copy of the License at
..
.. http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS,
.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. See the License for the specific language governing permissions and
.. limitations under the License.
Signature keys and user identity in libolm
==========================================
The use of any public-key based cryptography system such as Olm presents the
need for our users Alice and Bob to verify that they are in fact communicating
with each other, rather than a man-in-the-middle. Typically this requires an
out-of-band process in which Alice and Bob verify that they have the correct
public keys for each other. For example, this might be done via physical
presence or via a voice call.
In the basic `Olm <olm.html>`_ protocol, it is sufficient to compare the public
Curve25519 identity keys. As a naive example, Alice would meet Bob and ensure
that the identity key she downloaded from the key server matched that shown by
his device. This prevents the eavesdropper Eve from decrypting any messages
sent from Alice to Bob, or from masquerading as Bob to send messages to Alice:
she has neither Alice's nor Bob's private identity key, so cannot successfully
complete the triple-DH calculation to compute the shared secret, :math:`S`,
which in turn prevents her decrypting intercepted messages, or from creating
new messages with valid MACs. Obviously, for protection to be complete, Bob
must similarly verify Alice's key.
However, the use of the Curve25519 key as the "fingerprint" in this way makes
it difficult to carry out signing operations. For instance, it may be useful to
cross-sign identity keys for different devices, or, as discussed below, to sign
one-time keys. Curve25519 keys are intended for use in DH calculations, and
their use to calculate signatures is non-trivial.
The solution adopted in this library is to generate a signing key for each
user. This is an `Ed25519`_ keypair, which is used to calculate a signature on
an object including both the public Ed25519 signing key and the public
Curve25519 identity key. It is then the **public Ed25519 signing key** which is
used as the device fingerprint which Alice and Bob verify with each other.
By verifying the signatures on the key object, Alice and Bob then get the same
level of assurance about the ownership of the Curve25519 identity keys as if
they had compared those directly.
Signing one-time keys
---------------------
The Olm protocol requires users to publish a set of one-time keys to a key
server. To establish an Olm session, the originator downloads a key for the
recipient from this server. The decision of whether to sign these one-time keys
is left to the application. There are both advantages and disadvantages to
doing so.
Consider the scenario where one-time keys are unsigned. Alice wants to initiate
an Olm session with Bob. Bob uploads his one-time keys, :math:`E_B`, but Eve
replaces them with ones she controls, :math:`E_E`. Alice downloads one of the
compromised keys, and sends a pre-key message using a shared secret :math:`S`,
where:
.. math::
S = ECDH\left(I_A,\,E_E\right)\;\parallel\;ECDH\left(E_A,\,I_B\right)\;
\parallel\;ECDH\left(E_A,\,E_E\right)
Eve cannot decrypt the message because she does not have the private parts of
either :math:`E_A` nor :math:`I_B`, so cannot calculate
:math:`ECDH\left(E_A,\,I_B\right)`. However, suppose she later compromises
Bob's identity key :math:`I_B`. This would give her the ability to decrypt any
pre-key messages sent to Bob using the compromised one-time keys, and is thus a
problematic loss of forward secrecy. If Bob signs his keys with his Ed25519
signing key (and Alice verifies the signature before using them), this problem
is avoided.
On the other hand, signing the one-time keys leads to a reduction in
deniability. Recall that the shared secret is calculated as follows:
.. math::
S = ECDH\left(I_A,\,E_B\right)\;\parallel\;ECDH\left(E_A,\,I_B\right)\;
\parallel\;ECDH\left(E_A,\,E_B\right)
If keys are unsigned, a forger can make up values of :math:`E_A` and
:math:`E_B`, and construct a transcript of a conversation which looks like it
was between Alice and Bob. Alice and Bob can therefore plausibly deny their
partition in any conversation even if they are both forced to divulge their
private identity keys, since it is impossible to prove that the transcript was
a conversation between the two of them, rather than constructed by a forger.
If :math:`E_B` is signed, it is no longer possible to construct arbitrary
transcripts. Given a transcript and Alice and Bob's identity keys, we can now
show that at least one of Alice or Bob was involved in the conversation,
because the ability to calculate :math:`ECDH\left(I_A,\,E_B\right)` requires
knowledge of the private parts of either :math:`I_A` (proving Alice's
involvement) or :math:`E_B` (proving Bob's involvement, via the
signature). Note that it remains impossible to show that *both* Alice and Bob
were involved.
In conclusion, applications should consider whether to sign one-time keys based
on the trade-off between forward secrecy and deniability.
License
-------
This document is licensed under the `Apache License, Version 2.0
<http://www.apache.org/licenses/LICENSE-2.0>`_.
Feedback
--------
Questions and feedback can be sent to richard at matrix.org.
.. _`Ed25519`: http://ed25519.cr.yp.to/
Fuzzers
=======
This directory contains a collection of fuzzing tools. Each tests a different
entry point to the code.
Usage notes:
1. Install AFL:
.. code::
apt-get install afl
2. Build the fuzzers:
.. code::
make fuzzers
3. Some of the tests (eg ``fuzz_decrypt`` and ``fuzz_group_decrypt``) require a
session file. You can use the ones generated by the python test script
(``python/test.sh``).
4. Make some work directories:
.. code::
mkdir -p fuzzing/in fuzzing/out
5. Generate starting input:
.. code::
echo "Test" > fuzzing/in/test
6. Run the test under ``afl-fuzz``:
.. code::
afl-fuzz -i fuzzing/in -o fuzzing/out -- \
./build/fuzzers/fuzz_<fuzzing_tool> [<test args>]
7. To resume with the data produced by an earlier run:
.. code::
afl-fuzz -i- -o existing_output_dir [...etc...]
8. If it shows failures, pipe the failure case into
``./build/fuzzers/debug_<fuzzing_tool>``, fix, and repeat.
......@@ -54,6 +54,8 @@ int main(int argc, const char *argv[]) {
uint8_t plaintext[max_length];
uint32_t ratchet_index;
size_t length = check_error(
olm_inbound_group_session_last_error,
session,
......@@ -61,7 +63,7 @@ int main(int argc, const char *argv[]) {
olm_group_decrypt(
session,
message_buffer, message_length,
plaintext, max_length
plaintext, max_length, &ratchet_index
)
);
......
......@@ -140,7 +140,8 @@ size_t olm_group_decrypt(
uint8_t * message, size_t message_length,
/* output */
uint8_t * plaintext, size_t max_plaintext_length
uint8_t * plaintext, size_t max_plaintext_length,
uint32_t * message_index
);
......
......@@ -320,7 +320,8 @@ int olm_session_has_received_message(
/** Checks if the PRE_KEY message is for this in-bound session. This can happen
* if multiple messages are sent to this account before this account sends a
* message in reply. Returns olm_error() on failure. If the base64
* message in reply. Returns 1 if the session matches. Returns 0 if the session
* does not match. Returns olm_error() on failure. If the base64
* couldn't be decoded then olm_session_last_error will be "INVALID_BASE64".
* If the message was for an unsupported protocol version then
* olm_session_last_error() will be "BAD_MESSAGE_VERSION". If the message
......@@ -333,7 +334,8 @@ size_t olm_matches_inbound_session(
/** Checks if the PRE_KEY message is for this in-bound session. This can happen
* if multiple messages are sent to this account before this account sends a
* message in reply. Returns olm_error() on failure. If the base64
* message in reply. Returns 1 if the session matches. Returns 0 if the session
* does not match. Returns olm_error() on failure. If the base64
* couldn't be decoded then olm_session_last_error will be "INVALID_BASE64".
* If the message was for an unsupported protocol version then
* olm_session_last_error() will be "BAD_MESSAGE_VERSION". If the message
......
......@@ -96,7 +96,7 @@ size_t olm_init_outbound_group_session_random_length(
*/
size_t olm_init_outbound_group_session(
OlmOutboundGroupSession *session,
uint8_t const * random, size_t random_length
uint8_t *random, size_t random_length
);
/**
......
......@@ -403,8 +403,8 @@ DemoUser.prototype.decryptGroup = function(jsonpacket, callback) {
throw new Error("Unknown session id " + session_id);
}
var plaintext = session.decrypt(packet.body);
done(plaintext);
var result = session.decrypt(packet.body);
done(result.plaintext);
}, callback);
};
......
......@@ -73,10 +73,12 @@ InboundGroupSession.prototype['decrypt'] = restore_stack(function(
// So we copy the array to a new buffer
var message_buffer = stack(message_array);
var plaintext_buffer = stack(max_plaintext_length + NULL_BYTE_PADDING_LENGTH);
var message_index = stack(4);
var plaintext_length = inbound_group_session_method(Module["_olm_group_decrypt"])(
this.ptr,
message_buffer, message_array.length,
plaintext_buffer, max_plaintext_length
plaintext_buffer, max_plaintext_length,
message_index
);
// Pointer_stringify requires a null-terminated argument (the optional
......@@ -86,7 +88,10 @@ InboundGroupSession.prototype['decrypt'] = restore_stack(function(
0, "i8"
);
return Pointer_stringify(plaintext_buffer);
return {
"plaintext": Pointer_stringify(plaintext_buffer),
"message_index": Module['getValue'](message_index, "i32")
}
});
InboundGroupSession.prototype['session_id'] = restore_stack(function() {
......
{
"name": "olm",
"version": "1.3.0",
"version": "2.0.0",
"description": "An implementation of the Double Ratchet cryptographic ratchet",
"main": "olm.js",
"files": [
......
......@@ -328,7 +328,7 @@ def do_group_decrypt(args):
session = InboundGroupSession()
session.unpickle(args.key, read_base64_file(args.session_file))
message = args.message_file.read()
plaintext = session.decrypt(message)
plaintext, message_index = session.decrypt(message)
with open(args.session_file, "wb") as f:
f.write(session.pickle(args.key))
args.plaintext_file.write(plaintext)
......
......@@ -7,7 +7,7 @@ def read_random(n):
return f.read(n)
lib = cdll.LoadLibrary(os.path.join(
os.path.dirname(__file__), "..", "..", "build", "libolm.so.1")
os.path.dirname(__file__), "..", "..", "build", "libolm.so.2")
)
lib.olm_error.argtypes = []
......
......@@ -41,6 +41,7 @@ account_function(lib.olm_account_one_time_keys_length)
account_function(lib.olm_account_one_time_keys, c_void_p, c_size_t)
account_function(lib.olm_account_mark_keys_as_published)
account_function(lib.olm_account_max_number_of_one_time_keys)
account_function(lib.olm_pickle_account_length)
account_function(
lib.olm_account_generate_one_time_keys_random_length,
c_size_t
......
......@@ -43,6 +43,7 @@ inbound_group_session_function(
lib.olm_group_decrypt,
c_void_p, c_size_t, # message
c_void_p, c_size_t, # plaintext
POINTER(c_uint32), # message_index
)
inbound_group_session_function(lib.olm_inbound_group_session_id_length)
......@@ -82,11 +83,14 @@ class InboundGroupSession(object):
)
plaintext_buffer = create_string_buffer(max_plaintext_length)
message_buffer = create_string_buffer(message)
message_index = c_uint32()
plaintext_length = lib.olm_group_decrypt(
self.ptr, message_buffer, len(message),
plaintext_buffer, max_plaintext_length
plaintext_buffer, max_plaintext_length,
byref(message_index)
)
return plaintext_buffer.raw[:plaintext_length]
return plaintext_buffer.raw[:plaintext_length], message_index
def session_id(self):
id_length = lib.olm_inbound_group_session_id_length(self.ptr)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment