Remove tsd in favor of tsc + test utilities

[vkarpov15]

Summary

tsd is nice, but with the ts-go rewrite the future of tsd is uncertain. Either way, using TypeScript directly should be more accurate in terms of catching errors, and will hopefully allow us to test with multiple different TypeScript configs as well.

Examples

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[vkarpov15]

cc @chriskrycho

[AbdelrahmanHafez]

Couple of minor questions, overall LGTM.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[hasezoey]

Looks good to me, though some minor style questions.
tsd might not be able to keep working as-is when ts-go drops, as it ships its own typescript version and patches it. It also seems to practically be in maintenance mode (looking at tsdjs/tsd#224 not having a answer as of now).

Might it be worth to investigate switching to tstyche instead as that does not vendor typescript and only makes use of the typescript public API, which to my knowledge, they intend to keep working with ts-go?

[mrazauskas]

Might it be worth to investigate switching to tstyche instead

Thanks for mentioning TSTyche! (I am its author.) Yes, TypeScript 7 support is in the plans. I will start working on this as soon as APIs will be available.

---

Let me add some doubts regarding the following statement:

using TypeScript directly should be more accurate in terms of catching errors

In my opinion, // @ts-expect-error does a good job in code that will be executed, but not in type tests. The problem of // @ts-expect-error: it hides errors of missing imports, duplicate identifier, renamed APIs and anything else. These errors are introduced during refactoring and tsc cannot catch them. (A sad fact, I have seen a repository where removed APIs are still passing tests marked with // @ts-expect-error.)

In contrary, tsds expectError() is only accepting a predefined list of errors. Although on surface it might look similar to // @ts-expect-error.

TSTyche does even more. It is able to check errors suppressed by // @ts-expect-error (try it on StackBlitz).

There are many other differences between compiler (tsc) and type test runners. You know the requirements of your project. So use what suits you best.

[chriskrycho]

Just saw this! Will try to give it a gander in the next couple of days—thanks for the ping, and hooray for doing the work!

[hasezoey]

Still looks good to me, though i still dont quite like that ExpectAssignable requires a extra function invocation.

Also personally, i would still recommend with going to something like tstyche (mostly due to the concerns listed in #15951 (comment)).

[vkarpov15]

@mrazauskas I tried out tstyche, it is very cool. I like how you can assert on the error message with // @ts-expect-error.

Couple of questions:

1. How does tstyche work under the hood? From what I've been able to gather, it spawns tsc processes directly to compile individual test files and check the output, is that accurate?
2. How do I pass in test files to run? It seems to pick up tests I put in typetests dir just fine, but without a typetests then ./node_modules/.bin/tstyche ./test/types/*.test.ts fails with Error: No test files were selected using current configuration.
3. Can you bring your own assertions? expect<x>().toBe() is not my favorite syntax. Minor point, I'm more just curious.

[mrazauskas]

@vkarpov15 Thanks! Glad to hear that.

1. How does tstyche work under the hood? From what I've been able to gather, it spawns tsc processes directly to compile individual test files and check the output, is that accurate?

TSTyche uses TypeScript programmatically via language service API. This is the entry point used by IDEs and it is optimised to open / check a single file. Also the references of a project get handled. It works at any scale. In contrary, running tsc --noEmit does not work with references and always checks all files although lots of that job can be skipped.

.toBe(), .toBeAssignableTo() and .toBeAssignableFrom() compare types programmatically. Same as in tsd, but without patching TypeScript.

.toBeCallableWith(), .toBeConstructableWith(), .toHaveProperty() and other ability matchers are implemented via deleted syntax. A file gets open and checked for errors, if there are none, a virtual files gets open with deleted syntax and are errors get mapped to matchers. Here is how deleted syntax roughly looks like:

expect(getPerson).type.not.toBeCallableWith({ name: false })
;     (getPerson)                          ({ name: false })         

Having an error means the expression is not callable. (The matcher also makes sure that the expression has call signatures.)

Same with // @ts-expect-error. Errors are checked in a virtual file.

As you noticed, there are no generics in the matchers by design. Having any generics means the limitations of conditional types become limitations of the library.

2. How do I pass in test files to run? It seems to pick up tests I put in typetests dir just fine, but without a typetests then ./node_modules/.bin/tstyche ./test/types/*.test.ts fails with Error: No test files were selected using current configuration.

You can configure this via the testFileMatch option. Every file matched by the patterns is a test file. You can also run a single file: tstyche query-params.

By the way, test() and describe() helpers are provided. They are optional. .skip, .only can be used with the helpers as well as with assertions: expect.skip, expect.only.

3. Can you bring your own assertions? expect<x>().toBe() is not my favorite syntax. Minor point, I'm more just curious.

In case if it feels better, you can use TSTyche as a runner and bring in your own generic based assertions. That works. // @ts-expect-error are checked, you can run a single file or test against several versions of TypeScript.

I went with expect() style, because it allows passing expression or type on both sides of assertion. This means {} as unknown as SomeType casting is never needed.

At some point I plan to add assertType() (tstyche/tstyche#467). Would that look better? I need help / feedback regarding the syntax. The classical generic assertType<Expected>(actual) is somewhat odd, because expected goes before actual. Perhaps I am too conservative (; If it sounds interesting, lets talk in the issue.

[hasezoey]

Still looks good to me, I dont quite know what i should be reviewing since my last review though.

[vkarpov15]

@hasezoey that's my bad, I saw the "re-request review" icon on "Reviewers" so I just re-requested without reading your review comment, sorry