mini-redis outline proposal

[jxs]

Client

Along with the hello-world guide there should be a guide with the pubsub example, to give the user an idea on how the interaction with the server works on the client side

Server

• Start with TcpListener, explaining accept and spawning a new task for each connection.
• Implement Connection struct, describe the read_frame process and Frame conversion, introduce Bytes crate
• Implement Command struct and from_frame method explaining the Frame to Command parsing
• Add GET variant to Command and implement apply method introducing the internal Db and explaining the decisions behind it's implementation: why it's an Arc<Mutex>, the State and Entry struct's
• implement GET's write_frame and describe the process of writing a Command back to a Frame and writing it on the socket
• Add SET variant to Command enum, the flow is similar to GET, but has the key expiration logic. Guide through it's implementation with tokio's Duration and Notify
• Add PUB variant to Command and implement the Db pubsub behavior bringing up tokio's Broadcast struct
• Add SUB variant to Command, and implement the graceful shutdown notification logic of Server
• Implement the client connection limit handling on the Server.
• Introduce tokio-tracing and instrument the code

[carllerche]

Thanks for getting this started. I have taken time to think about this proposal and it definitely was helpful to get the ball going.

My initial thought was to structure guides like you outlined. However, after sitting on it some, I think it would be better to structure it around tokio concept to teach.

So, we would figure out the order in which we want to introduce Tokio concepts and pick the component from mini-redis that covers that concept.

I started a little bit with this in the guides:

Outline

• Basic async (async/await)
  • redis client set / get
• Spawning
  • Server accept loop
• Intro to concurrency
  • Shared DB state (shared via std mutex).
• Common sync primitives: mpsc, oneshot
  • client connection pool.
• I/O basics
• Framing
  • implement Connection
• select!, join!, try_join!
  • [Question]: Should this be batched w/ graceful shutdown?
  • join! / try_join! used to run redis client commands concurrently.
• Cancellation via drop
  • Cancel reads via drop.
  • Cancel channels via drop.
• Timer
  • More robust accept loop w/ back-off
• Streams
  • ??? how to cover combinators / iteration
• Graceful shutdown
• Broadcast channel
• Error handling??
• Bridging sync w/ async
  • blocking mini-redis client.

Topics that should be in tutorial

• How to not leak tasks.

[Darksonn]

Summary from discussion on discord, using #12 as base for ideas to maybe add.

We should have a section about "When should you use async/await? (and when should you not?)" as one of the first things people read. That said, it should probably go in the Tokio overview document instead of the tutorial.

Interfacing sync w/ async should probably go somewhere near the end. This fits quite nicely with building a client, as we can reproduce a reqwest::blocking style module. This would also cover how to explicitly construct a Runtime object.

The select! macro would be covered in graceful shutdown and again in broadcast channel. It should also be mentioned when we talk about join! and try_join!.

The join! and try_join! macros should be illustrated by performing multiple redis commands concurrently. This should also talk about simulating join_all by spawning a bunch of times, and how join_all is not necessary if you also use tokio::spawn, i.e. you can just for loop over the join handles. This should also mention join_all and FuturesUnordered from the futures crate, unless we add our own.

We sprinkle "futures as objects" all over the tutorial. They should see this example. I'm not quite sure how much detail we should go in, but I want a paragraph like this somewhere:

Imagine you had a list of one thousand elements called futures. Each future has a method called poll, which does a little bit of work and then returns quickly. By looping through the list and calling poll on each future, you can do work on all one thousand tasks at once in a single thread.

from here

And this could be related to join! and explain how it is possible for join! to not use multiple threads.