When should you write an error if you have no output?

[Jaykul]

There's been some discussion recently about when it's appropriate to write errors ...

1. A bug filed against Get-TypeData
2. The PR to fix the "bug"
3. A bug filed against Get-Module

On behalf of the PowerShell Committee, @SteveL-MSFT wrote this:

@PowerShell/powershell-committee discussed this and we also discussed it in this morning's Community Call. We agreed that the intent is that if a literal is passed and is not found, it should return a non-terminating error. Cmdlets that don't have this behavior have bugs.

In cases where it's explicit or implicit filtering, it should return nothing.

The real issue is that we are missing Test-* cmdlets so users had to rely on current behavior with Get-* cmdlets to see if something exists and we should add those cmdlets in the future.

And then later clarified:

• the intent is for a literal to return a non-terminating error, but wildcards/filtering to return nothing
• we do not plan to accept changes to existing behavior to make them consistent, but any new cmdlets should adhere to this
• even without changing the existing cmdlets, we still want to introduce Test-* cmdlets and make that the norm and best practice
• if not documented, we need to document existing cmdlet behavior where it deviates from the intent

So I ask this community: is that right?

Should a Get command that takes a literal name always write an error when it does not return anything?

Pretend that PowerShell was green-field, and that whatever you decide here will not break existing code.

Would you want errors from commands like:

1. Get-Module ModuleThatIsNotLoaded
2. Get-Module ModuleThatIsNotInstalled -ListAvailable
3. Get-ChildItem FolderThatDoesNotExist
4. Get-ChildItem EmptyFolder
5. Get-TypeData System.Boolean # there's no TypeData for this by default

What about listing commands where we expect to always have data, but there are edge cases:

6. Get-Command -Verb Build # there aren't any commands that use this verb yet
7. Get-History # As the first command in a session
8. Get-Clipboard # If the clipboard is empty

[Jaykul]

Spoiler: I think this is wrong.

Here is the clearest, most obvious counter-example I can think of right now: Get-AzureKeyVaultSecret

I use this command because it's one that has a few easy to understand use cases and specific issues that lead to a specific design (in my opinion). It has a few parameter sets:

1. If you don't pass a name, it will output ALL the secrets
2. If you pass a name, it will output only that one secret (the latest version of it, or a specified version of it), except ...
3. If you specify -IncludeVersions (instead of a specific version) it will output the entire history of the specific thing
4. You can extend your search to deleted items

In each case, if it doesn't find a matching value, it quietly returns nothing. I consider that to be the best possible design, because:

• As a web command, it's likely to encounter connectivity problems, resulting in terminating errors
• As a command that accesses data that is secure and has access controls, it's likely to encounter permission errors even when the data exists.

However, any time you're looking up an item by name, it's very likely that the value doesn't exist (yet). If the command produced an error for the "everything worked fine, but there's nothing to return" case, I believe it would be extremely complicated to use correctly, and would generally result in careless or inexperienced users hiding the important errors.

Error handling for these cases is terrible

The first time a user calls the command and it returns a trivial "not found" error, they might choose to suppress that error by adding -ErrorAction Ignore or SilentlyContinue and would then miss out on the errors that are actually important (i.e. connectivity problems, or access errors like "Operation 'list' is not allowed by vault policy").

A design which produces errors for missing values forces careful users to capture the error and check which error it was. In fact ...

if the error is not terminating, it's even worse than if it were terminating like the others.

In order to suppress a non-terminating error, you must use -ErrorAction and either:

• set it to SilentlyContinue (which also suppresses terminating exceptions) and capture the error, test the result, and throw if it's not the simple error
• set it to Stop and wrap it in try/catch in order to test the error type, ignore the "does not exist" error and rethrow the rest.

However, even if the command simply throws a different unique exception for each use case (or writes an error with a different unique error category), producing an error in this situation results in the user needing to explicitly handle the error or avoid it ...

In order to allow you to avoid the error, the PowerShell Committee's proposal essentially requires every Get command to have a Test command.

For expensive commands, I'm afraid this this is a terrible idea

If Get-AzureKeyVaultSecret wrote errors for missing entries and had a separate Test command, then in order to avoid the very complicated error handling for the Get command, you would need to instead wrap your Get call in a Test call.

The problem is that the Test call would require the same slow and heavy round-trip, authenticated network request, and would still have all the same exception cases for access and networking errors.

Worse yet, you're not just making TWO requests when you could just make one -- it's actually the exact same request each time, because there's no REST method for "test" so you would be calling the GET method for Test and for Get ... returning the full data each time ...

[SQLDBAWithABeard]

On phone so employing brevity.

I strongly disagree that there should be an error.

I see

Get-ChildItem EmptyFolder

As

Give me all of the contents of the folder.

If there aren't any don't return anything. It's not an error if it is doing what I ask.

I also think it is misleading as "an error" means something is wrong. Noth in ng to return is not "wrong"

[TimCurwick]

I strongly agree with Joel.

Returning nothing when there is nothing to return is the appropriate response and not an error.

[rkeithhill]

I don't think Get-ChildItem EmptyFolder is in question here. What the PS committee says is this:

We agreed that the intent is that if a literal is passed and is not found ...

Well, EmptyFolder was found but it was empty. So this proposal does NOT suggest that this scenario should generate a non-terminating error. However the following does error:

PS> Get-ChildItem xyzzy
Get-ChildItem : Cannot find path 'C:\Users\hillr\xyzzy' because it does not exist.
...

And I think rightly so. CMD and Bash also error in this case.

And while we're looking at these examples ... Get-Clipboard isn't a command on PowerShell Core and even on Windows PowerShell, it only takes switch params. So I'm not sure how it applies to this proposal. Get-Command -Verb build is a filtering command that doesn't error. Check. Get-Command xyzzy looks for a specific command and does generate a non-terminating error. Check (and good). Get-History does not error when you run it right after PS starts - when there is no history yet. Check. However it fails if you pass it an id that doesn't exist Get-History -id 8675309. Check (also good).

RE 1, this is a filtering scenario therefore it is behaving as designed - to not return a non-term error. Personally I'm not big fan of these "inherently filtering" commands. If I wanted filtering I would just specify Get-Module ModuleThatIsNotLoaded*. Then I would have more control (filtering or not). Also, there are cases where I want an error when a module is not found. Now, I have to generate that myself rather than rely on -EA stop. Oh well. Too late to make any changes to this command.

RE 2, doesn't error now and wouldn't error if implemented under the proposal IMO. The use of -ListAvailable makes this a filtering scenario.

RE 5, it would have been nice if this generated a non-terminating error IMO. Again would have been easy to avoid with Get-TypeData System.Boolean* or Get-TypeData System.Boolean -ea 0. But "probably" not worth changing at this point.

Back to the original question:

When should you write an error if you have no output?

IMO you error when you don't have the specific item I asked for and your design is to only return a single item i.e. not all things that match the term I specified. But please, require a wildcard char for filtering - that's what they're for. However, there are nuances here, I'll give you that. Like asking for the contents of a specific (empty) folder should not generate an error. Or asking for the "contents" of a specific (empty) file should not generate an error either.

[rkeithhill]

they might choose to suppress that error by adding -ErrorAction Ignore or SilentlyContinue and would then miss out on the errors that are actually important (i.e. connectivity problems ...

Just because a command writes non-terminating errors doesn't mean it can't also throw terminating errors. A connectivity problem should result in a terminating error since there's no way the command can proceed.

[SQLDBAWithABeard]

I can see that there are strong views for this to continue as originally specified but now I have access to a proper keyboard I will respond again.

This is the conundrum

If

Get-ChildItem EmptyFolder
Get-Module ModuleThatIsNotLoaded

should not return an error when there is nothing there

why should

Get-Command NoneExisitngCommand

return an error when there is no command of that name?
I have asked for this particular command and I have received an error rather than either

a, Here is precisely what you asked for - nothing
b, A more useful message to the user than a full error message

I disagree that this should be an error. I also find that it confuses newcomers. An error and a sea of red text means something is wrong and in this case nothing is wrong, it just doesn't exist.

Personally, I want it to return nothing because there is nothing to be returned. I would be happy with a warning but I see it as a direct question that should be return in a pedantic manner.

"Give me all of the Printers named foo please"

"Here are all of the printers named foo "

Funnily enough 5 minutes afer writing this I find another example that maybe also explains why I see it his way. This is what I get with the virtualmachinemanager module right now too.

Give me all of the Virtual Machines named $Vmname
(There aren't any - we are cleaning up a system of old data prior to upgrade)

Which makes my coding easier and training the users easier also ;-)

It makes it easier for users at the command line and it makes it easier for error handling too as Joel mentions above more eloquently than I can.

Thats my opinion

[Jaykul]

@rkeithhill but you've ignored my argument in favor of picking apart the op examples. let's set all of that aside. Read my first reply, and explain to me how your logic applies to Get-AzureKeyVaultSecret (after that, we can talk about all these other examples from a point of understanding each other, rather than just defending the current behavior).

However, this comment seems like a HUGE misunderstanding, which in my opinion is the root cause of your point of view (and everyone who thinks these should error):

Just because a command writes non-terminating errors doesn't mean it can't also throw terminating errors. A connectivity problem should result in a terminating error since there's no way the command can proceed.

That's simply not reasonable. Take this sample command:

function Test-Error {
[CmdletBinding()]
param(
    [switch]$Exception,
    [switch]$Error
)
if($Error){
    Write-Error "Test Error" -Category SecurityError
}

if($Exception) {
    throw ([Exception]::new("Syntax Problem"))
}

}

Are you saying that the user should just learn to ignore and live with the huge block of red from Write-Error, or do you have a magic incantation whereby they can suppress it without suppressing the exception?

[rkeithhill]

do you have a magic incantation whereby he can suppress it without suppressing the other?

Yup:

PS>  try { Test-Error -Error -Exception -ea 0 } catch { "oops" }
# << Look Ma, no red (non-terminating) error text
oops

Or if you prefer:

PS> try { Test-Error -Error -Exception } catch {}; Get-Date
Test-Error : Test Error
At line:1 char:7
+ try { Test-Error -Error -Exception } catch {}; Get-Date
+       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Test-Error


Wednesday, August 22, 2018 8:53:45 AM

@SQLDBAWithABeard then you should be disappointed in a lot of PowerShell Get- commands: Get-Variable, Get-Process, Get-PSDrive, Get-PSRepository, Get-PackageSource, Get-PackageProvider, Get-DscResource, etc, etc, etc.

[RamblingCookieMonster]

I'm on board with what @Jaykul wrote up top. Tweet-sized:

I much prefer no error. Testing for absence is incredibly easy, can be used in various logic statements, is the same for every command. Looking for a specific error type indicating something isn't found? Ugly af, and IMHO goes against the tenets of PowerShell

The existing commands that follow the error-if-nothing-found convention are painful to deal with.

I don't understand why this is even up for debate. Do folks seriously want to create a common convention forcing everyone who writes PowerShell to handle multiple error types (which differ for each command, generally) for something that should (and often is, currently) as simple as testing for presence of a value? I realize some folks here are very experienced, but this seems like a bad idea that would affect the usability and desirability of the language for a large swath of folks, including myself.

[rkeithhill]

I wonder if it is just the "sea of red" that so many folks have an aversion to that they're willing to not have commands error just to avoid it. I can certainly sympathize. We've tried in the past to argue for more succinct error msgs. I would love to see something like this:

PS> try { Test-Error -Error -Exception } catch {}
Test-Error : Test Error
For more details on this error run: $error[0]
<optional error hook to allow an extensible error message hint capability>

What I don't like about that is that the $error collection is a stack and that error index above of 0 is not stable. I'd like to see something more like this:

PS> try { Test-Error -Error -Exception } catch {}
Test-Error : Test Error
For more details on this error run: Get-Error 8c1c74a
<optional error hook to allow an extensible error message hint capability>

where 8c1c74a uniquely identifies the error record and can be used at any time up until that error record is "aged" out of the $error collection. That would look like:

PS> Get-Error 8c1c74a

Test-Error : Test Error
At line:1 char:7
+ try { Test-Error -Error -Exception } catch {}; Get-Date
+       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Test-Error

Also, with Get-Error, I'd like to see a -Verbose parameter where I could get extended error details - much like the Resolve-Error (in PSCX) command provides:

Get-Error 8c1c74a -Verbose

PSMessageDetails      :
Exception             : System.Exception: Syntax Problem
TargetObject          :
CategoryInfo          : OperationStopped: (:) [], Exception
FullyQualifiedErrorId : Syntax Problem
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at Test-Error, <No file>: line 13
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}

MyCommand             :
BoundParameters       : {}
UnboundArguments      : {}
ScriptLineNumber      : 13
OffsetInLine          : 5
HistoryId             : -1
ScriptName            :
Line                  :     throw ([Exception]::new("Syntax Problem"))

PositionMessage       : At line:13 char:5
                        +     throw ([Exception]::new("Syntax Problem"))
                        +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PSScriptRoot          :
PSCommandPath         :
InvocationName        :
PipelineLength        : 0
PipelinePosition      : 0
ExpectingInput        : False
CommandOrigin         : Internal
DisplayScriptPosition :

Exception at nesting level 0 ---------------------------------------------------

Message        : Syntax Problem
Data           : {}
InnerException :
TargetSite     :
StackTrace     :
HelpLink       :
Source         :
HResult        : -2146233088

[rkeithhill]

Do folks seriously want to create a common convention forcing everyone who writes PowerShell to handle multiple error types

Yup. Although the "forcing everyone" bit sounds ominous. Did you see that list I posted above? You already have to do this with many Get-* commands. Are you suggesting we should change those and incur breaking changes?

Also folks seem to be only considering the scripting case and not the interactive CLI case. I happen to believe that most folks using a shell expect Get-Item xyzzy to error - just not with a sea of red text. :-) It's been that way on Bash for decades, ditto for CMD.

[RamblingCookieMonster]

@rkeithhill or others who are in the it should error when you request a specific item and that item is not found - can you spell out a value proposition here? I'm struggling to see one. bash does this or some-random-command does this is not really a strong value here, IMHO, when this convention adds serious overhead

[rkeithhill]

can you spell out a value proposition here?

Given that the system temp dir is C:\Windows\Temp (and you are running elevated), what do you expect this to return:

PS> gci C:\Windows\Tmp

If this command were to return nothing, I would assume the directory is empty and not that it actually doesn't exist. That's not a good user experience.

[SQLDBAWithABeard]

@SQLDBAWithABeard then you should be disappointed in a lot of PowerShell Get- commands: Get-Variable, Get-Process, Get-PSDrive, Get-PSRepository, Get-PackageSource, Get-PackageProvider, Get-DscResource, etc, etc, etc.

Yes - when considering this subject.

Although I can work around it when coding or when at the command line I can read it. I have to explain it to many people who are intimidated by the sea of red. I always want PowerShell to be as easy as possible for all people

If there is a chance to influence the choice, my choice is for no error if nothing to return

Given that the system temp dir is C:\Windows\Temp (and you are running elevated), what do you expect this to return:

PS> gci C:\Windows\Tmp
If this command were to return nothing, I would assume the directory is empty and not that it actually doesn't exist. That's not a good user experience.

I think this is a poor example @rkeithhill GCI Folderthatdoesntexist is asking for the contents of a folder that doesnt exist so that should give some output

Get-Printer Doesntexist
Get-AzureRmVM -Name NotThere -ResourceGroup $RG

Should not error IMHO

[rkeithhill]

This command can be easily made to behave the way you want:

Get-Printer Doesntexist*

BTW I think you guys are tilting at windmills here. On my Windows 10 1803 system in an elevated Windows PowerShell (more commands built-in), I count the following number of get commands:

gcm -verb get | ? {$_.ParameterSets.Parameters.Name -eq 'Name' -and $_.Name -notin "Get-ItemProperty","Get-WebConfigurationProperty","Get-WmiObject"} | 
    measure

Count    : 41

Of those 41, 29 of them error when the specified item is not found:

$error.clear()
gcm -verb get | 
    ? {$_.ParameterSets.Parameters.Name -eq 'Name' -and $_.Name -notin "Get-ItemProperty","Get-WebConfigurationProperty","Get-WmiObject"} | 
    % {& $_ -Name xyzzy}
$error.Count

That's 71% of the built-in Get commands that do error when the specified item is not found. So even if you don't think the PowerShell Committee is right on this (and yeah, they do have a few years experience), I don't see them changing over to "not finding the specified item should not (non-term) error". That said, I do get @Jaykul perf argument for REST calls. But again, I think for new REST/remote commands like this, you just specify * to tell the command to not write a non-term error if the wildcard specifier results in no output.

Update: the above results are highly dependent on what modules you have loaded. And it also only checks Get commands with a Name parameter. I tried this again with the just following modules loaded:

Microsoft.PowerShell.Management
Microsoft.PowerShell.Security
Microsoft.PowerShell.Utility
Microsoft.WSMan.Management
PackageManagement
PowerShellGet
PrintManagement
PSReadline
PSScheduledJob
PSWorkflow
WebAdministration

And got 41/43 errored which is 95% that error. I'm sure the % will vary depending on the modules you have loaded.