Jon Gjengset: Rust for Rustaceans

Rules/guidelines

Writing functions

• Borrowed vs owned argument:
  • if you need ownership of data (call methods that take self, move data to another thread), must store data. Make the caller provide owned data, because then caller is in control of allocation, and the allocation is known upfront.
  • if you don’t need ownership, operate on references instead.
  • if not always, use the Cow type – operate on references if data allows, produce an owned data if needed.
• Static/dynamic dispatch: use static dispatch (generics) in libraries – users can use dynamic dispatch if they want. Use dynamic dispatch (&dyn Trait) in binaries, because cleaner code with fewer generic params at usually marginal performance cost.
• use generic rather than concrete types, to ask for arguments based on what your function needs instead of concrete types (e.g. impl AsRef<str> instead of &str). Start with fully generic argument with no bounds, then follow compiler errors to add bounds.
• for arguments only taken by reference, consider replacing generic argument with dynamic dispatch (instead of impl AsRef<str>, use &dyn AsRef<str>). But users cannot opt out and must always provide trait object, so not a great idea if code may be performance-sensitive.
• If any part of your interface exposes foreign types, then any change to one of those foreign types is also a change to your interface. To mitigate issues like this, it’s often best to wrap foreign types using the newtype pattern, and then expose only the parts of the foreign type that you think are useful. In many cases, you can avoid the newtype wrapper altogether by using impl Trait to provide only the very minimal contract to the caller. By promising less, you make fewer changes breaking.
• With Result, when you have an Ok or Err that you know will never be used, you can set it to the ! type. If you write a function that returns Result<T, !>, you will be unable to ever return Err, since the only way to do so is to enter code that will never return.
• A try block acts pretty much like a single-iteration loop, where ? uses break instead of return, and the final expression of the block has an implicit break.

Generic traits

• Associated type wherever you can. Use associated type if you only expect one trait implementation for a given type, use generic type parameter otherwise.

Writing types

• always implement Debug, Clone, Default

• PartialEq trait is particularly desirable, because users will at some point inevitably have two instances of your type that they wish to compare with == or assert_eq!. Even if your type would compare equal for only the same instance of the type, it’s worth implementing PartialEq to enable your users to use assert_eq!.

• serde Serialize/Deserialize: these can be easily derived, and the serde_derive crate even comes with mechanisms for overwriting the serialization for just one field or enum variant. Since serde is a third-party crate, you may not wish to add a required dependency on it. Most libraries therefore choose to provide a serde feature that adds support for serde only when the user opts into it.

• Deref: If you provide a relatively transparent wrapper type (like Arc), there’s a good chance you’ll want to implement Deref so that users can call methods on the inner type by just using the . operator

• It’s good practice to include some simple tests in your test suite that check that all your types implement these traits the way you expect. This does not run any code, but if it doesn’t compile, auto-trait implementations are broken.

  fn is_normal<T: Sized + Send + Sync + Unpin>() {}
  #[test]
  fn normal_types() {
      is_normal::<MyType>();
  }
  

Writing errors

• When you write code that can fail, the most important question to ask yourself is how your users will interact with any errors returned
• creating error types:
  • implement std::error::Error. The main method of interest is Error::source, which provides a mechanism to find the underlying cause of an error.
  • if possible, implement Send and Sync, so error can be shared across thread boundaries
  • if possible, error type should be 'static. Allows caller to more easily propagate error type without lifetime issues, and enabled more easy usage with type-erased error types.
    • Downcasting, which only works for 'static, allows user to turn dyn Error into concrete underlying type, using Error::downcast_ref
  • implement Display and Debug (required if you implement Error)
    • Display should give a one-line description of what went wrong that can easily be folded into other error messages. The display format should be lowercase and without trailing punctuation so that it fits nicely into other, larger error reports.
    • Debug should provide a more descriptive error including auxiliary information that may be useful in tracking down the cause of the error, such as port numbers, request identifiers, filepaths, and the like, which #[derive(Debug)] is usually sufficient for.
• With Box<dyn Error>, you leave your users with little option but to bubble up your error
• You should usually avoid “simplifying” a Result<T, ()> to Option<T>. An Err(()) indicates that an operation failed and should be retried, reported, or otherwise handled exceptionally. None, on the other hand, conveys only that the function has nothing to return; it is usually not considered an exceptional case or something that should be handled. You can see this in the #[must_use] annotation on the Result type—when you get a Result, the language expects that it is important to handle both cases, whereas with an Option, neither case actually needs to be handled.

Documentation

• Use #[doc(cfg(..))] to highlight items that are available only under certain configurations so the user quickly realizes why some method that’s listed in the documentation isn’t available.
• Use #[doc(alias = "...")] to make types and methods discoverable under other names that users may search for them by

Logging

tracing crate

Dependency management

• Set up CI infrastructure to perform basic auditing of your dependencies using tools like cargo-deny and cargo-audit. These tools will detect cases where you transitively depend on multiple major versions of a given dependency, where you depend on crates that are unmaintained or have known security vulnerabilities, or where you use licenses that you may want to avoid.
• List the earliest version that has all the things your crate depends on and to make sure that this remains the case even as you add new code to your crate. Your best bet is to use Cargo’s unstable -Zminimal-versions flag, which makes your crate use the minimum acceptable version for all dependencies, rather than the maximum. Then, set all your dependencies to just the latest major version number, try to compile, and add a minor version to any dependencies that don’t. Rinse and repeat until everything compiles fine, and you now have your minimum version requirements!

Macros vs generics

If your code changes based on type, use generics. Otherwise, use macros.

Compiling and testing

• Check undefined behavior with cargo miri test
• Enable extra compiler warnings: rust_2018_idioms, missing_docs, missing_debug_implementation

Other notes

Concurrency/multithreading

Standardized polling:

• in Rust, polling is standardized through the Future trait
• Types that implement the Future trait are known as futures and represent values that may not be available yet. A future could represent the next time a network packet comes in, the next time the mouse cursor moves, or just the point at which some amount of time has elapsed. You can read Future<Output = Foo> as “a type that will produce a Foo in the future.” Types like this are often referred to in other languages as promises
• These nonblocking functions allow us to easily perform multiple tasks concurrently. For example, if you want to read from either the network or the user’s keyboard, whichever has an event available first, all you have to do is poll both in a loop until one of them returns Poll::Ready. No need for any additional threads or synchronization!

Why not threads: hard to keep track, threads add up fast and complexity to keep track does too, switching between many threads is costly (each switch is round-trip to OS scheduler), introduce parallelism

Concurrency: execution of tasks is interleaved

Parallelism: multiple tasks executing at the same time

Macros

Declarative macros:

• consist of two main parts: matchers and transcribers. A given macro can have many matchers, and each matcher has an associated transcriber.
• must be declared before use. Even in lib.rs – the order of mod is important. If you mark macro with #[macro_export], the macro is hoisted to root of crate and marked as pub, so it can be used anywhere. Procedural macros:
• types:
  • derive (like #[derive(Serialize)]): automate implementation of a trait where possible (only adds to target of macro)
  • attribute (like #[test]): replaces item, takes as input the token tree in the attribute and the token tree of the entire item
  • function-like macro: replaces macro code at call site with returned code. No protection from interacting with identifiers in surrounding code, not hygienic. Instead, your macros are expected to explicitly call out which identifiers should overlap with the surrounding code (using Span::call_site) and which should be treated as private to the macro (using Span::mixed_site)
• can significantly increase compilation: bring heavy dependencies (like syn, try to disable features you don’t need, compile procedural macros in debug mode), make it easy to generate a lot of code without realizing
• TokenStream type, iterate to get individual TokenTree items
• TokenTree: single token or another TokenStream enclosed in a delimiter
• by walking a TokenStream, parse out whatever syntax you want with syn crate (can turn TokenStream into Rust AST)
• every TokenTree also has a span – ties generated code back to source code
• to produce rust code to be injected into program: either manually construct TokenStream and extend one TokenTree at a time, or use TokenStream’s FromStr (with "".parse::<TokenStream>()).

Traits

Orphan rule: you can implement a trait for a type only if the trait or the type is local to your crate. You can’t implement Debug for bool.

Generics

Functions

Static dispatch (fn x(a: impl Pattern), fn x<T: Pattern>(a: T)): need different copy of x for each impl Pattern type – need to know address of instance functions to call them. For each of the types, create a copy of the method with right addresses.

• use in libraries. Allow users to choose; if you use dynamic dispatch, users have to use dynamic too.

Dynamic dispatch (fn x(a: &dyn Pattern)): the caller must give address of pattern and address of any methods called on it, via vtable.

• why &? We no longer know at compile time the size of the pattern type (dyn Trait is not Sized), but we know the size of pointers/references.
• use in binaries, often leads to cleaner code (fewer generic parameters) at a usually marginal performance cost.

Traits

Rust traits can be generic: with type parameters (trait Foo<T>) or with associated types (trait Foo { type Bar;}). Use associated type if you only expect one implementation of the trait for a given type, use generic type parameter otherwise. Just use associated types whenever you can.

Trait object: type that implements a trait, and its vtable. So, dyn Trait is a trait object.

• sealed trait is one that can be used only, and not implemented, by other crates

Type system guidance

• Marker types: consider a type like SshConnection, which may or may not have been authenticated yet. You could add a generic type argument to SshConnection and then create two marker types: Unauthenticated and Authenticated. When the user first connects, they get SshConnection<Unauthenticated>. In its impl block, you provide only a single method: connect. The connect method returns a SshConnection<Authenticated>, and it’s only in that impl block that you provide the remaining methods for running commands and such.

• Use an equal sign in the type parameter for default

• We introduce unit types to represent each stage of the rocket. We don’t actually need to store the stage—only the meta-information it provides—so we store it behind a PhantomData to guarantee that it is eliminated at compile time. Then, we write implementation blocks for Rocket only when it holds a particular type parameter.

  struct Grounded;
  struct Launched;
  struct Rocket<Stage = Grounded> {
      stage: std::marker::PhantomData<Stage>,
  }
  
  impl Default for Rocket<Grounded> {}
  impl Rocket<Grounded> {
      pub fn launch(self) -> Rocket<Launched> { }
  }
  impl Rocket<Launched> {
      pub fn accelerate(&mut self) { }
      pub fn decelerate(&mut self) { }
  }
  
  impl<Stage> Rocket<Stage> {
      pub fn color(&self) -> Color { }
      pub fn weight(&self) -> Kilograms { }
  }
  

• with #[must_use] on type/trait/function, compiler issues a warning if user code receives an element of that type/trait or calls function, and does not explicitly handle it (like unhandled Result). Add it only if the user is very likely to make a mistake if they are not using the return value.

Testing:

• Integration tests (the tests in tests/) follow the same process as unit tests, with the one exception that they are each compiled as their own separate crate, meaning they can access only the main crate’s public interface and are run against the main crate compiled without #[cfg(test)].
• Because doctests appear in the public documentation of your crate, and users are likely to mimic what they contain, they are run as integration tests. This means that the doctests don’t have access to private fields and methods, and test is not set on the main crate’s code.
• To explore fuzzing further, I recommend starting with cargo-fuzz.
• If you’re curious about property-based test generation, I recommend starting with the proptest crate
• Running cargo miri test runs through mid-level intermediate representation, which is slower, but reports if program exhibits undefined behavior (uninit memory reads, use after drop, out-of-bounds pointer access).
• Loom: ensure tests run with every relevant interleaving of concurrent operations
  • can catch only bugs that manifest as panics – use assertions
  • Loom expects you to write dedicated test cases in the form of closures that you pass into a Loom model. The model keeps track of all cross-thread interactions and tries to intelligently explore all possible iterations of those interactions by executing the test case closure multiple times.
• Sanitizers: ThreadSanitizer (TSan), AddressSanitizer (ASan)

Not all compiler warnings are enabled by default. Those disabled by default are usually still being refined, or are more about style than content. A good example of this is the “idiomatic Rust 2018 edition” lint, which you can enable with #![warn(rust_2018_idioms)]. When this lint is enabled, the compiler will tell you if you’re failing to take advantage of changes brought by the Rust 2018 edition. Some other lints that you may want to get into the habit of enabling when you start a new project are missing_docs and missing_debug_implementations, which warn you if you’ve forgotten to document any public items in your crate or add Debug implementations for any public types, respectively.

With std::hint::black_box(x), you can tell compiler to assume that x is used in arbitrary ways that can’t be optimized out.