Can't stop busy instance

[RudolfMan]

Thank you for this library! We are moving project from using Bypass to TestServer.

Here is the simple scenario to reproduce the exit of the instance that causes test to crash:

defmodule Test do
  use ExUnit.Case

  test "exit" do
    {:ok, server} = TestServer.start()
    
    TestServer.add(server, "/", to: fn conn ->
      # pretend that the server is slow
      Process.sleep(10_000)
      Plug.Conn.send_resp(conn, 200, "OK")
    end)
    
    # call the server in the background
    Task.start(fn -> Req.get!(TestServer.url(server), receive_timeout: 15_000) end)
    
    Process.sleep(100) # just to make sure that the previous Task has started
    
    # stop the server
    TestServer.stop(server)

    # test should pass in here
  end
end

it ends up with

     ** (exit) exited in: GenServer.call(#PID<0.756.0>, :options, 5000)
         ** (EXIT) time out
     stacktrace:
       (elixir 1.17.2) lib/gen_server.ex:1128: GenServer.call/3
       (test_server 0.1.20) lib/test_server/instance_manager.ex:28: TestServer.InstanceManager.stop_instance/1

How I ended up there?
I'm trying to accomplish an equivalent of "stub", when 0 or more requests to the server is allowed and it doesn't verify if any request is made. Like just incase if my test happen to make the request I wan't it to return, no matter how many times and I'm not concerned about it.

For now I wrote a helper like:

  def stub(server, path, opts) do
    # suppose 10 is plenty for my stubs.. otherwise I could allow to customize it via opts or something
    for _ <- 1..10 do
      TestServer.add(server, path, opts)
    end

    # shut it down so it doesn't check if any of the expectations are not met
    ExUnit.Callbacks.on_exit(fn ->
      try do
        TestServer.stop(server)
      rescue
        _ -> :ok
      end
    end)
  end

I figured I could catch the :exit and terminte the instance from the InstanceSupervisor

   def stub(server, path, opts) do
     # suppose 10 is plenty for my stubs.. otherwise I could allow to customize it via opts or something
     for _ <- 1..10 do
       TestServer.add(server, path, opts)
     end

     # shut it down so it doesn't check if any of the expectations are not met
     ExUnit.Callbacks.on_exit(fn ->
       try do
         TestServer.stop(server)
       rescue
         _ -> :ok
+      catch
+       :exit, _reason ->
+         DynamicSupervisor.terminate_child(TestServer.InstanceSupervisor, server)
+         :ok
      end
     end)
   end

But it pauses the test for 5sec to catch the :exit and also I guess it leaves TestServer.HTTPServer hanging(?), which is not nice either.

[danschultzer]

It's because you are working against some expectations in TestServer. TestServer works as FIFO, and you don't want to block any of the requests. You can only do simultaneous request by spinning up multiple instances

It's just a GenServer going through messages being passed, and the stop comes after the requests so it'll first finish up the request messages, and then handle stop at the very end. That's why it times out because it's in the middle of dealing with the request message(s).

Usually if I just need to stub I would do it on the HTTP client rather than calling an actual endpoint as it just need to be as fast as possible. What is it you are testing here that requires to make the request take longer than the test itself?

Now with that said, I think it might still be a good idea to not block the GenServer once it processes the request, and just pass that off to some async task instead. For now, if you want to just get past this, and you do need to have a server that accepts arbitrary requests with no care for the responses I would recommend starting the webserver youself with a plug.

With Bandit it's all just:

Bandit.start_link(plug: TestPlug)

And set up a simple plug module for this.