Most importantly, differentiate between library and non-library components. The former can be provided and required, and if provided but disabled, can be replaced by pre-installed versions. The latter can only be provided and cannot be the target of a dependency relationship.
The caveat is needed because if you include one of these headers on a system that defines rsize_t without having the correct incantations in your configure.ac, you are going to have a bad time. But there is no good general solution to the problem.
On systems that have strlcat() and strlcpy() in libc, run the tests twice (once with our implementation and once with the system's) to verify that our tests are correct.
It was wrong to remove $(AM_CPPFLAGS) in d43a6bf2, because it is only used for code for which there is no explicit *_CPPFLAGS. It is not entirely clear why this did not trip us (or Travis) up until now, although it is possible that it only breaks when $(builddir) != $(srcdir).
On the other hand, there is no reason to use $(INCLUDES).
The count we passed to memcmp() in mpi_eq() and mpi_eq_abs() was actually the number of significant words in the MPI, rather than the number of bytes we wanted to compare. Multiply by 4 to get the correct value.
To make the intent of the code more apparent, introduce a private MPI_MSW() macro which evaluates to the number of significant words (or 1-based index of the most significant word). This also comes in handy in mpi_{add,sub,mul}_abs().
Add a couple of test cases which not only demonstrate the bug we fixed here but also demonstrate why we must compare whole words: on a big-endian machine, we would be comparing the unused upper bytes of the first and only word instead of the lower bytes which actually hold a value...
We failed to clear the negative flag when handling trivial cases, so if one of the terms was 0 and the other was negative, the result would be an exact copy of the non-zero term instead of its absolute value.
- Use the new vector byte-order conversion functions where appropriate.
- Use memset_s() instead of memset() where appropriate.
- Use consistent names and types for function arguments.
- Reindent, rename and reorganize to conform to Cryb style and idiom.
SHA224 and SHA256 were left mostly unchanged. MD2 and MD4 were completely rewritten as the previous versions (taken from XySSL) seem to have been copied from RSAREF.
This breaks the ABI as some context structures have grown or shrunk and some function arguments have been changed from int to size_t.
We now have separate encryption and decryption methods, and can process an arbitrary amount of plaintext or ciphertext per call, rounded down to the block size (if applicable). For stream ciphers, we also have a keystream method which fills the provided buffer with an arbitrary amount of keystream (once again, rounded down if applicable).