Series Outline Link to heading

  1. Overview
  2. Framework analysis and selection: Delving into the specifics of framework selection and application.
    1. Core Components: Database management, modularization, internal RPC, AMQP.
    2. Web and API: Web Server, GraphQL API.
    3. Application Development: Web, CLI, desktop, and mobile app development.
    4. Miscellaneous Tools: Observability, logging, machine learning.
  3. Boilerplate project: A comprehensive guide including setup instructions for selected frameworks, architectural diagrams, and development environment configurations.

ML/stats Link to heading

CrateDownloadsDependentsGithub StarsGithub ContributorsGithub Used ByNotes
polars500k8821k3024.3kDataframes for data prep/filtering/etc - this is easily the best choice. Can now do streaming, so, can handle > in memory as well.
linfa162k243k37321decent set of algorithms. OpenBLAS, netlib, intel MKL (all linux)
rustlearn16k15998121Not nearly as robust as linfa, but does have some unique algorithms.

Logging/Metrics Link to heading

CrateDownloadsDependentsGithub StarsGithub ContributorsGithub Used ByNotes
log197M147701.9k102605kDefault choice. syslog, systemd, slog, android, windows, database, console, and a lot more.
tracing109M49674.4k277241kCompatible with log, which is nice. metrics, tracing, and logging. wasm support.

Distributed Locking using Redis Link to heading

Redis is assumed for caching - but for dist lock, we have lots of options…

CrateDownloadsDependentsGithub StarsGithub ContributorsGithub Used ByNotes
redlock34k04611?port of the ruby version. seems fine.
redsync9k012?Good, if we want a sync interface.
rust_redlock4.7k017??also seems fine, but, not supported anymore.
rslock2.4k02313?Good async interface. Decent support, but not popular?

Summary Link to heading

ML Link to heading

Data prep - easy choice with polars. Seems like default choice for ML should be linfa, but, if that doesn’t support what we need to do, go look for specific algos.

Logging Link to heading

Log crate is fine, but tracing is much more flexible (3 for 1 deal - logging, tracing, and metrics), well supported by other libraries we’ve talked about (tower, hyper, tonic, axum, etc, etc), and has gained plenty of traction. Opentelemetry via tracing-opentelemetry is also recommended, which gets you all sorts of nice stuff.

Locking Link to heading

Seems to me we’ll want to wrap up our own helper(s) around this, none of the libraries I could find are “great” - so, we should pick one, and wrap it for the way(s) we want to use it. Perhaps we write a macro?