Multicasting chat with encryption (ECDH P-256 + AES-256-CBC)

Few years ago I published simple Python sample code for the multicasting chat, and this article is a long overdue follow-up that takes that sample to a completely new level.

Current IT industry is unimaginable without cybersecurity features, and trends like "zero-trust" rely on complicated crypto algorithms which could be quite overwhelming for beginners. The example C++ code I’ve published could help you to get the ideas and code snippets for your own projects. The full source code is available on Gitlab.

DISCLAIMER

The code is provided as is, and has issues (including code duplication) as it mostly serves as an education project, not the production-ready code that could benefit from the proper class hierarchies, edge cases handling and better code coverage.

First, let’s review the main phases of the application:

  1. Parse command line options (-v - verbose output, -e - apply encryption)

  2. Generate EC (elliptic curve, NIST P-256 in the code) keypair. They usually call those keys ephemeral in a context of using the keypair for the current session only.

  3. Send the public key to other peers for ECDH (Elliptic-curve Diffie–Hellman) key agreement. During that process they use their own private key and a third party public key to generate a shared secret. That secret could be used to create a key that is used for payload encryption/decryption between the host and the third party.

  4. Usually you can’t use a shared secret as is, but rather have to derive a key from it. The code uses SHA-256 hash, but it’s recommended to use KDF (Key Derivation Function) like PBKDF2.

  5. Send the chat messages encrypted with AES-256-CBC using the key generated above. Please note that this cipher requires IV (Initialization Vector) that is included into the sent payload. It’s not always necessary though (not all ciphers require it), plus some KDF procedures generate (key, IV) pairs that could be used together.

Second, let me describe the communication protocol, or rather two types of messages a peer sends:

  • REGISTER: R.BASE64(<name>).BASE64(<public_key>). The purpose of the message is informing other peers about the public key that is used in ECDH key agreement (step 3 above). The message is sent in two use-cases:

    • Starting the chat. That also serves as a new peer notification to other peers in the local network.

    • Upon getting notification from other peers (i.e. if a REGISTER message with the new <name> has been received). As a new peer doesn’t know yet the public keys of other peers, we need to resend the public key to that peer. Generally speaking, that should be the unicast message, but for the brevity we reuse the same multicast protocol.

  • MESSAGE: M.BASE64(<name>).BASE64(<IV>).BASE64(ENCRYPT(<message>, <key1>))…​BASE64(ENCRYPT(<message>, <keyN>)). That type is used to actually send the chat messages. As each peer has its own shared secret, we need to use all secrets to encrypt the chat message, thus all those keyi encrypted payloads.

Compilation, usage and test output:

$ clang++ chat.cc -lcrypto -pthread -o chat
$ ./chat -ev

Enter your name: alex
Encryption cipher: AES-256-CBC
Private key information:
   Private-Key: (256 bit)
   ASN1 OID: prime256v1
   NIST CURVE: P-256
Public key information:
   Public-Key: (256 bit)
   ASN1 OID: prime256v1
   NIST CURVE: P-256
Start listening...
Raw public key:
   30 59 30 13 06 07 2a 86 48 ce 3d 02 01 06 08 2a
   86 48 ce 3d 03 01 07 03 42 00 04 d1 f8 4e 2b 91
   97 b3 96 35 bb 02 dc 6d 26 a0 63 09 e2 b1 00 da
   5a 0a 80 27 7a 24 f8 10 d6 99 5a a0 cd 6e 66 6f
   3d 95 d0 ea 17 4c 9a da 8d 50 be ad 8f b9 5b 89
   7c a3 27 13 60 fb e0 35 62 e2 a2
-> R.YWxleA==.MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE0fhOK5GXs5Y1uwLcbSagYwnisQDaWgqAJ3ok+BDWmVqgzW5mbz2V0OoXTJrajVC+rY+5W4l8oycTYPvgNWLiog==
<- REGISTER alex KEY: MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE0fhOK5GXs5Y1uwLcbSagYwnisQDaWgqAJ3ok+BDWmVqgzW5mbz2V0OoXTJrajVC+rY+5W4l8oycTYPvgNWLiog==
<- Raw public key:
   30 59 30 13 06 07 2a 86 48 ce 3d 02 01 06 08 2a
   86 48 ce 3d 03 01 07 03 42 00 04 d1 f8 4e 2b 91
   97 b3 96 35 bb 02 dc 6d 26 a0 63 09 e2 b1 00 da
   5a 0a 80 27 7a 24 f8 10 d6 99 5a a0 cd 6e 66 6f
   3d 95 d0 ea 17 4c 9a da 8d 50 be ad 8f b9 5b 89
   7c a3 27 13 60 fb e0 35 62 e2 a2
test message
-> M.YWxleA==.23ifu2ciV+W6XH/pxxGfEg==.np2rmVf5Cddngs1JBFVTqA==.
<- MESSAGE alex IV: 23ifu2ciV+W6XH/pxxGfEg==
<- PAYLOAD: np2rmVf5Cddngs1JBFVTqA==
alex: test message

You could see REGISTER and MESSAGE messages above (and a successful attempt to decode PAYLOAD). tcpdump (started earlier) shows that the traffic is encrypted indeed:

$ sudo tcpdump -A 'udp port 6543'

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on wlp2s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:45:54.431179 IP amber.59578 > 224.0.0.100.lds-distrib: UDP, length 135
E...m.@...h........d.......KR.YWxleA==.MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE0fhOK5GXs5Y1uwLcbSagYwnisQDaWgqAJ3ok+BDWmVqgzW5mbz2V0OoXTJrajVC+rY+5W4l8oycTYPvgNWLiog==
19:45:56.636165 IP amber.59578 > 224.0.0.100.lds-distrib: UDP, length 61
E..Yn.@...g3.......d.....E..M.YWxleA==.23ifu2ciV+W6XH/pxxGfEg==.np2rmVf5Cddngs1JBFVTqA==.

If few peers are involved, MESSAGE contains several payloads. From another peer (gary) side (sending):

message from gary
-> M.Z2FyeQ==.NgQe5SgHCvZWYmwKhy0lYw==.xwAArSbH8ASK1zfor9ekLQTdLoniv7k1YLnjgwE6lpc=.tor5juICo7QKClU3Qapism4km80sxgkAx6zYQrdpmfw=.

And local peer (alex) side (receiving):

<- MESSAGE gary IV: NgQe5SgHCvZWYmwKhy0lYw==
<- PAYLOAD: xwAArSbH8ASK1zfor9ekLQTdLoniv7k1YLnjgwE6lpc=
gary: message from gary
<- PAYLOAD: tor5juICo7QKClU3Qapism4km80sxgkAx6zYQrdpmfw=

As you could see above there are several payload decryption attempts (and one successful).

And now let’s explore the actual snippets. I used OpenSSL (libcrypto) and POSIX functions (mostly, for socket programming). The code has been tested in GNU/Linux (other platforms may require few tweaks). Multithreading is implemented using native C++ functionality (please note that it requires C++11 and higher support).

Few notes about the code:

  • _(title, fn) is the series of functions to help with the error handling, you may ignore them except the fn part.

  • A lot of API uses std::string for the convenience only. Please consider those data types as containers for bytes. In production code you may want to use std::vector<uint8_t> or even OpenSSL’s BIO structures directly.

Base64 encoding/decoding functions (base64 namespace):

  std::string encode(const std::string& data) {
    std::shared_ptr<BIO> b64(BIO_new(BIO_f_base64()), BIO_free_all);
    BIO_set_flags(b64.get(), BIO_FLAGS_BASE64_NO_NL);
    auto bio = BIO_new(BIO_s_mem());
    BIO_push(b64.get(), bio);
    BIO_write(b64.get(), data.c_str(), data.size());
    BIO_flush(b64.get());

    char *str;
    auto size = BIO_get_mem_data(bio, &str);
    return std::string(str, size);
  }

  std::string decode(const std::string& data) {
    std::shared_ptr<BIO> b64(BIO_new(BIO_f_base64()), BIO_free_all);
    BIO_set_flags(b64.get(), BIO_FLAGS_BASE64_NO_NL);
    auto bio = BIO_new_mem_buf(data.c_str(), data.size());
    BIO_push(b64.get(), bio);
    std::shared_ptr<BIO> bio_out(BIO_new(BIO_s_mem()), BIO_free_all);

    char inbuf[512];
    int inlen;
    while ((inlen = BIO_read(b64.get(), inbuf, sizeof(inbuf))) > 0 ) {
      BIO_write(bio_out.get(), inbuf, inlen);
    }
    BIO_flush(bio_out.get());

    char *str;
    auto size = BIO_get_mem_data(bio_out.get(), &str);
    return std::string(str, size);
  }

Please note that OpenSSL has EVP_EncodeUpdate/EVP_DecodeBlock helper functions, but apparently they don’t use BIO_FLAGS_BASE64_NO_NL, therefore the encoded text has PEM format (with new lines) that could be not compatible with some Base64 parsers.

Generate EC keypair (crypto namespace):

  std::shared_ptr<EVP_PKEY> generate_pair(int nid) {
    std::shared_ptr<EVP_PKEY_CTX>
      pctx(_("EVP_PKEY_CTX_new_id",
             EVP_PKEY_CTX_new_id(EVP_PKEY_EC, nullptr)),
           EVP_PKEY_CTX_free);

    _("EVP_PKEY_paramgen_init", EVP_PKEY_paramgen_init(pctx.get()));

    _("EVP_PKEY_CTX_set_ec_paramgen_curve_nid",
      EVP_PKEY_CTX_set_ec_paramgen_curve_nid(pctx.get(), nid));

    EVP_PKEY *params_ptr = nullptr;
    _("EVP_PKEY_paramgen", EVP_PKEY_paramgen(pctx.get(), &params_ptr));
    std::shared_ptr<EVP_PKEY> params(params_ptr, EVP_PKEY_free);

    std::shared_ptr<BIO> out(BIO_new_fp(stdout, BIO_NOCLOSE), BIO_free);
    if (verbose) {
      std::cout << "Private key information:" << std::endl;
      EVP_PKEY_print_private(out.get(), params.get(), 3, NULL);
      std::cout << "Public key information:" << std::endl;
      EVP_PKEY_print_public(out.get(), params.get(), 3, NULL);
    }

    std::shared_ptr<EVP_PKEY_CTX>
      kctx(_("CLT: EVP_PKEY_CTX_new",
             EVP_PKEY_CTX_new(params.get(), nullptr)),
           EVP_PKEY_CTX_free);

    _("EVP_PKEY_keygen_init", EVP_PKEY_keygen_init(kctx.get()));

    EVP_PKEY *pkey_ptr = nullptr;
    _("EVP_PKEY_keygen", EVP_PKEY_keygen(kctx.get(), &pkey_ptr));
    return std::shared_ptr<EVP_PKEY>(pkey_ptr, EVP_PKEY_free);
  }

The input parameter is NID_X9_62_prime256v1 (that is NIST P-256 curve). Output EVP_PKEY structure contains both public and private keys.

Extract public key from EVP_PKEY structure (crypto namespace):

  std::string extract_public_key(const std::shared_ptr<EVP_PKEY> &pkey) {
    std::shared_ptr<BIO> bio(BIO_new(BIO_s_mem()), BIO_free_all);
    _("i2d_PUBKEY_bio", i2d_PUBKEY_bio(bio.get(), pkey.get()));

    char *str;
    auto size = BIO_get_mem_data(bio.get(), &str);

    std::string public_key(str, size);
    if (verbose) {
      string::dump("Raw public key:", public_key);
    }
    return public_key;
  }

That public key is being sent to other peers. While it’s not required to escape the binary data for the UDP datagrams, we use Base64 encoding on top of them for the debugging purpose (e.g. the verbose output to the console).

Derive shared key (crypto namespace):

  std::string derive_key(const std::shared_ptr<EVP_PKEY> pkey,
                         const std::string &public_key) {
    typedef std::vector<unsigned char> uc_vector;

    std::shared_ptr<BIO> decoded(BIO_new(BIO_s_mem()), BIO_free_all);
    BIO_write(decoded.get(), public_key.c_str(), public_key.size());

    EVP_PKEY *peerkey_ptr = nullptr;
    _("d2i_PUBKEY_bio", d2i_PUBKEY_bio(decoded.get(), &peerkey_ptr));
    std::shared_ptr<EVP_PKEY> peerkey(peerkey_ptr, EVP_PKEY_free);

    std::shared_ptr<EVP_PKEY_CTX>
      ctx(_("SRV: EVP_PKEY_CTX_new",
            EVP_PKEY_CTX_new(pkey.get(), nullptr)),
          EVP_PKEY_CTX_free);

    _("EVP_PKEY_derive_init", EVP_PKEY_derive_init(ctx.get()));

    _("EVP_PKEY_derive_set_peer",
      EVP_PKEY_derive_set_peer(ctx.get(), peerkey.get()));

    size_t secret_len = 0;
    _("EVP_PKEY_derive length",
      EVP_PKEY_derive(ctx.get(), nullptr, &secret_len));
    uc_vector secret(secret_len);
    _("EVP_PKEY_derive",
      EVP_PKEY_derive(ctx.get(), &secret[0], &secret_len));

    uc_vector hash(SHA256_DIGEST_LENGTH);
    SHA256(&secret[0], secret_len, &hash[0]);
    return std::string(hash.begin(), hash.end());
  }

pkey is the host’s keypair, and public_key is the public key of the third party peer. The derived secret is hashed using SHA256 helper function and the resulting hash is used as a key for symmetric encryption/decryption.

Symmetric encryption/decryption (crypto namespace):

  std::string encrypt(const std::string& plaintext,
                      const EVP_CIPHER *cipher,
                      const std::string& key,
                      const std::string& iv) {
    typedef std::vector<unsigned char> uc_vector;

    std::shared_ptr<EVP_CIPHER_CTX> ctx(_("EVP_CIPHER_CTX_new",
                                          EVP_CIPHER_CTX_new()),
                                        EVP_CIPHER_CTX_free);

    uc_vector uc_key(key.begin(), key.end());
    uc_vector uc_iv(iv.begin(), iv.end());
    _("EVP_EncryptInit_ex",
      EVP_EncryptInit_ex(ctx.get(), cipher, nullptr,
                         &uc_key[0], &uc_iv[0]));

    int len = plaintext.size() + EVP_CIPHER_block_size(cipher);
    uc_vector ciphertext(len);
    uc_vector uc_plaintext(plaintext.begin(), plaintext.end());
    _("EVP_EncryptUpdate",
      EVP_EncryptUpdate(ctx.get(),
                        &ciphertext[0], &len,
                        &uc_plaintext[0], uc_plaintext.size()));

    int tmplen = 0;
    _("EVP_EncryptFinal_ex",
      EVP_EncryptFinal_ex(ctx.get(), &ciphertext[len], &tmplen));

    ciphertext.resize(len + tmplen);
    return std::string(ciphertext.begin(), ciphertext.end());
  }

  std::string decrypt(const std::string& ciphertext,
                      const EVP_CIPHER *cipher,
                      const std::string& key,
                      const std::string& iv) {
    typedef std::vector<unsigned char> uc_vector;

    std::shared_ptr<EVP_CIPHER_CTX> ctx(_("EVP_CIPHER_CTX_new",
                                          EVP_CIPHER_CTX_new()),
                                        EVP_CIPHER_CTX_free);

    uc_vector uc_key(key.begin(), key.end());
    uc_vector uc_iv(iv.begin(), iv.end());
    _("EVP_DecryptInit_ex",
      EVP_DecryptInit_ex(ctx.get(), cipher, nullptr,
                         &uc_key[0], &uc_iv[0]));

    uc_vector uc_ciphertext(ciphertext.begin(), ciphertext.end());
    int len = ciphertext.size() + EVP_CIPHER_block_size(cipher);
    uc_vector plaintext(len);
    _("EVP_DecryptUpdate",
      EVP_DecryptUpdate(ctx.get(), &plaintext[0], &len,
                        &uc_ciphertext[0], uc_ciphertext.size()));

    int tmplen;
    // we expect failures as we try to decrypt using different keys
    if (!EVP_DecryptFinal_ex(ctx.get(), &plaintext[len], &tmplen)) {
      return std::string();
    }

    plaintext.resize(len + tmplen);
    return std::string(plaintext.begin(), plaintext.end());
  }

cipher is a result of EVP_aes_256_cbc() call (AES-256-CBC). key length could be retrieved using EVP_CIPHER_key_length(cipher), iv length - using EVP_CIPHER_iv_length(cipher) call.

Multicast socket programming is straightforward, you could review the code by yourself. Let me just add few notes here:

  • Please ensure to use a proper multicast address (i.e. ffx2::/16 for IPv6 and 224.0.0.0/24 for IPv4 are used for the local subnetwork only), the sample app uses 224.0.0.100.

  • IP_ADD_MEMBERSHIP is required to receive multicast datagrams (please review all setsockopt usages in the app).

That concludes the review of the encryption-enabled networking application. Surely, it’s rather a sandbox than the real production code, but it could be a starting point to the next secure product of yours. Happy hacking!

Comments

Post a Comment

Popular posts from this blog

Web application framework comparison by memory consumption

Memory consumption is slightly specific to my area of software development now, but I did some research recently and maybe these results can be useful for others. Of course, I know that precious comparison is very difficult to carry out, but actually I needed only overall picture. And let me admit that results are pretty interesting and even frustrated (at least for me). As a basis I took so-starving project, and measured initial RSS (Resident Set Size) of the each process (local development webservers). Platform: x86_64 Linux (latest Ubuntu with all updates). As a reference, here is the RSS of the interpreters in interactive console mode: Interpreter Version RSS (kB) stackless python 2.6.4 3916 ruby (via irb) 1.8.7 4664 python 2.7.1 5624 php 5.3.5 6924 v8 (via node.js shell) 2.5.9.9 8796 One more reference - the most simplest WSGI app ( example  in the Python documentation). It's RSS: 7336 Kb , so I assume it's almost impossible to consume ...
Read more

Trac Ticket Workflow

Trac is the wiki and issue tracking system that is the most used for my software development projects. Let me share few hints about its ticket workflow, especially about the statuses, the roles of the participants, and let me give an example of the workflow config. But if you are interested in other things like Trac plugins, source browser, customizing, etc, let me know. All the information given below regards the workflow described at the end of the article. The original Trac workflow is missed vital features like the support for ticket verifying, or the required ticket statuses range, so I do not use it in my projects. The description of the main statuses: new - the ticket has been created; in-progress - the developer has began to work with the ticket; fixed - the ticket is fixed (the problem issued in the ticket is solved); verified - the ticket is verified (tester or PM has verified that the ticket is fixed correctly); closed - the ticket is closed (it's not...
Read more

Python vs JS vs PHP for embedded systems

UPDATE (07/18/17): The original article was written in 2011 and pretty much outdated, I've updated the numbers and conclusions. I've got a question about which programming language is preferable for the website development for embedded systems (with limited resources). Here is my small investigation in a table form. Please note that the question was about only these 3 programming languages - there are better candidates for the embedded systems now (for example, Rust). The Memory and Performance overhead numbers are based on the n-body benchmark and calculated as relatives to "C gcc #4" measurements. Python 3 Node.js PHP Memory Overhead x7.62 x29.04 x8.53 Type System Strong Typing Weak Typing Weak Typing Vulnerabilities 220 1411 5626 Performance Overhead x78.31 x2.82 x30.12 Documentation Excellent Excellent Average C Bindings Excellent Average Poor Code Readability With PEP8 it can be almost perfect ESLint enforces very good style PEAR Coding Stan...
Read more