The objects of the last lesson came from the file system. The same commands (Get-ChildItem, Get-Item, Get-Content) also work with completely different things: environment variables, the registry on Windows, the functions of the session. Providers make that possible. In this lesson you get to know them, read and write files correctly (encoding, paths) and process the two formats you meet most often: CSV and JSON.

Providers and drives

A provider makes a data store accessible as if it were a drive. Each drive has a name and belongs to a provider:

PowerShell
Get-PSDrive -PSProvider FileSystem, Environment | Select-Object Name, @{ n = 'Provider'; e = { $_.Provider.Name } } |
    Where-Object Name -in 'Env', 'C' | Sort-Object Name
Name Provider
---- --------
C    FileSystem
Env  Environment

(The drive C exists only on Windows; on Linux and macOS the list looks different, not run.) The drive Env: shows the environment variables, and the ordinary commands work there:

PowerShell
$env:NORDLICHT_STUFE = 'test'
Get-Item Env:NORDLICHT_STUFE | Select-Object Name, Value
Name            Value
----            -----
NORDLICHT_STUFE test

$env:NAME is the shorthand for reading and setting. The variable applies only to this session and the processes it starts. Get-PSDrive without parameters lists all drives, including Function: (the functions of the session) and Variable:. On Windows HKLM: and HKCU: for the registry are added. The pattern is always: whoever knows the commands for files knows them for all providers.

Paths: the .NET trap

PowerShell knows a current directory: Set-Location (alias cd) sets it, Get-Location reads it. The cmdlets resolve relative paths against this directory. Calls straight to .NET do not do that: they resolve relative paths against the working directory of the process, and that is not what Set-Location set.

PowerShell
Set-Location $root
[IO.File]::WriteAllText('relativ.txt', 'x')
Test-Path (Join-Path $root 'relativ.txt')
Test-Path (Join-Path ([Environment]::CurrentDirectory) 'relativ.txt')
[Environment]::CurrentDirectory -eq (Get-Location).Path
False
True
False

The file is not in $root, where Set-Location pointed, but in the working directory of the process ([Environment]::CurrentDirectory). The rule that follows: in calls of .NET methods always use absolute paths:

PowerShell
[IO.File]::WriteAllText((Join-Path $PWD 'richtig.txt'), 'x')
Test-Path (Join-Path $root 'richtig.txt')
True

$PWD is the variable for the current PowerShell directory, Join-Path assembles path parts with the right separator. For the cmdlets too: scripts that run on a server do not rely on where they were started. They build paths from a parameter or from $PSScriptRoot (the folder of the script, more in the next lesson).

Two more concerns belong here. -Path understands wildcards such as * and [ ]; whoever takes a file name from data that might contain such characters uses -LiteralPath so that nothing is interpreted as a pattern. Tried: a file [a].txt is read by Get-Content -LiteralPath (result inhalt), while Get-Content -Path with the same name returns nothing, because [a] counts as a pattern. And Test-Path asks whether something exists before you access it.

Reading and writing files, encoding

PowerShell
$p = Join-Path $root 'umlaut.txt'
Set-Content $p 'Grüße aus Hamburg'
$b = [IO.File]::ReadAllBytes($p)
($b[0..2] | ForEach-Object { '{0:X2}' -f $_ }) -join ' '
$b.Length
Get-Content $p
47 72 C3
21
Grüße aus Hamburg

The first three bytes are 47 72 C3: G, r and the first byte of ü in UTF-8. There is no byte order mark in front. That is the default of PowerShell 7 for Set-Content: UTF-8 without BOM. In Windows PowerShell 5.1 the default is a different one. The same run with a ü (typed as the character 0xFC so that the source file plays no role):

7: 47 72 C3 BC 0D 0A
5: 47 72 FC 0D 0A

PowerShell 7 writes C3 BC (UTF-8, two bytes), Windows PowerShell 5.1 writes FC (the ANSI encoding of the system language, one byte). Both lines end with 0D 0A, because on Windows Set-Content appends a line break. Whoever exchanges files between the two worlds fixes the encoding instead of relying on the default:

PowerShell
Set-Content $p 'Grüße' -Encoding utf8BOM

Valid values are among others utf8 (without BOM), utf8BOM, utf8NoBOM, unicode, ascii. As a rule of thumb: UTF-8 without BOM, stated explicitly, wherever you create and read the file. A Windows PowerShell 5.1 script whose source file is saved as UTF-8 without BOM is read by 5.1 as ANSI, and umlauts in the script itself come out wrong; a BOM in the script file prevents that.

Line endings: Get-Content reads a file line by line (an array of strings) and knows \r\n and \n. With -Raw you get the whole content back as one string, line endings included:

PowerShell
Set-Content $p "a`r`nb`r`nc" -NoNewline
(Get-Content $p).Count
(Get-Content $p -Raw).Length
3
7

Three lines, but seven characters in all: a, b, c and twice \r\n. Whoever reads a file with Windows line endings on a Linux machine or the other way round should know this when comparing strings. The backtick ` is in PowerShell the character for escape sequences in double quotes (`r is carriage return, `n line feed, more in the next lesson).

CSV

CSV is the exchange format of tables. Import-Csv reads a file into objects, a column becomes a property:

PowerShell
$csv = Join-Path $root 'buchungen.csv'
@'
kunde,reise,preis
Nordlicht,Oslo,420.50
Nordlicht,Bergen,380
Fjord,Tromsø,999.9
'@ | Set-Content $csv
$rows = Import-Csv $csv
$rows[0].GetType().Name
$rows[0].preis.GetType().Name
PSCustomObject
String

Each row is a PSCustomObject, and every value is a string, the number too. For comparisons and calculations you convert explicitly ([decimal]):

PowerShell
$rows | Where-Object { [decimal]$_.preis -gt 400 } | Select-Object kunde, reise
($rows | ForEach-Object { [decimal]$_.preis } | Measure-Object -Sum).Sum
kunde     reise
-----     -----
Nordlicht Oslo
Fjord     Tromsø

1800,4

(again with the decimal comma of this machine). Without the conversion -gt would compare strings, and '380' -gt '400' sorts by character, not by value. Export-Csv writes objects back:

PowerShell
$rows | Export-Csv (Join-Path $root 'aus.csv') -NoTypeInformation
Get-Content (Join-Path $root 'aus.csv')
"kunde","reise","preis"
"Nordlicht","Oslo","420.50"
"Nordlicht","Bergen","380"
"Fjord","Tromsø","999.9"

All fields are in quotes. In PowerShell 7 Export-Csv no longer writes a type line (-NoTypeInformation does no harm there but is not needed); in Windows PowerShell 5.1 without the switch the first line is #TYPE System.Management.Automation.PSCustomObject (run that way).

JSON

JSON is the format of web interfaces and many configuration files. ConvertTo-Json turns objects into text:

PowerShell
$o = [pscustomobject]@{ name = 'Nordlicht'; reisen = @(@{ ziel = 'Oslo'; tage = 3 }); aktiv = $true }
$o | ConvertTo-Json -Depth 3
{
  "name": "Nordlicht",
  "reisen": [
    {
      "ziel": "Oslo",
      "tage": 3
    }
  ],
  "aktiv": true
}

The parameter -Depth is the trap: by default ConvertTo-Json goes only two levels deep and cuts off below. With four nested objects, the innermost level is then replaced by its text form:

PowerShell
$deep = [pscustomobject]@{ a = [pscustomobject]@{ b = [pscustomobject]@{ c = [pscustomobject]@{ d = 1 } } } }
$deep | ConvertTo-Json -Compress
$deep | ConvertTo-Json -Compress -Depth 5
{"a":{"b":{"c":"@{d=1}"}}}
{"a":{"b":{"c":{"d":1}}}}

At the first call PowerShell also prints a warning that the JSON was truncated (in German on this machine). Whoever overlooks it writes incomplete configuration. The rule: always give -Depth when the objects can be nested. The reverse direction reads text into objects:

PowerShell
$j = '{"name":"Nordlicht","reisen":[{"ziel":"Oslo","tage":3}]}' | ConvertFrom-Json
$j.reisen[0].ziel
Oslo

Regular expressions and Select-String

The operator -match checks a string against a regular expression and, if it matches, fills the automatic variable $Matches:

PowerShell
'bericht-2025-11.txt' -match '(\d{4})-(\d{2})'
$Matches[1]
$Matches[2]

\d{4} means “four digits”, parentheses form groups: $Matches[1] is 2025, $Matches[2] is 11. Select-String searches files; it returns hits with the line number:

PowerShell
Set-Content (Join-Path $root 'log.txt') @('INFO ok', 'ERROR kaputt', 'INFO ok', 'ERROR noch einmal')
Select-String -Path (Join-Path $root 'log.txt') -Pattern '^ERROR' | ForEach-Object { '{0}: {1}' -f $_.LineNumber, $_.Line }
2: ERROR kaputt
4: ERROR noch einmal

^ anchors at the start of the line. More about the patterns is in about_Regular_Expressions on Learn.

Web interfaces: Invoke-RestMethod

Invoke-RestMethod calls an HTTP address and, when the answer is JSON, returns objects right away. A local test so that you need no network: a tiny server in the session answers once with JSON.

PowerShell
$l = [System.Net.HttpListener]::new()
$l.Prefixes.Add('http://localhost:8765/'); $l.Start()
$job = Start-ThreadJob { param($l) $c = $l.GetContext(); $b = [Text.Encoding]::UTF8.GetBytes('{"status":"ok","anzahl":3}'); $c.Response.ContentType = 'application/json'; $c.Response.OutputStream.Write($b, 0, $b.Length); $c.Response.Close() } -ArgumentList $l
$r = Invoke-RestMethod http://localhost:8765/
$r.GetType().Name
$r.anzahl
$job | Wait-Job | Remove-Job; $l.Stop()
PSCustomObject
3

The answer is already a PSCustomObject, $r.anzahl is 3. The example uses Start-ThreadJob to let the server answer in the background; you do not have to understand that for everyday work. With real interfaces you simply write Invoke-RestMethod -Uri … and, where needed, give -Headers and -Method. Secrets (tokens) never belong on the command line, which can end up in a log: lesson PS.4.2 deals with that.

Exercise 1: The path trap

  1. Create a directory, move into it with Set-Location and write a file with [IO.File]::WriteAllText('a.txt','x'). Where does it lie? Check with Test-Path.
  2. Now write it correctly with Join-Path $PWD 'b.txt'.
  3. How do you read the file [a].txt without [a] counting as a pattern?

Solution 1

  1. It lies in the working directory of the process ([Environment]::CurrentDirectory), not in the new folder.
  2. [IO.File]::WriteAllText((Join-Path $PWD 'b.txt'), 'x') lands in the folder.
  3. Get-Content -LiteralPath '[a].txt'.

Exercise 2: Encoding

  1. Write 'Grüße' to a file with Set-Content, read the bytes with [IO.File]::ReadAllBytes and check whether a BOM (EF BB BF) is at the front.
  2. Write it again with -Encoding utf8BOM and compare.
  3. Which encoding do you choose for a file that another team reads on another system, and why do you state it explicitly?

Solution 2

  1. In PowerShell 7 there is no BOM (47 72 C3 …).
  2. With utf8BOM the file begins with EF BB BF.
  3. UTF-8, explicitly: then the result does not depend on the default of the version and the system language.

Exercise 3: CSV and JSON

  1. Create the file buchungen.csv from the section and compute the sum of the prices. Why do you need [decimal]?
  2. Turn the rows into text with ConvertTo-Json. How many levels are there, and which -Depth do you need at least?
  3. Read the text back with ConvertFrom-Json and print the destination of the first booking.

Solution 3

  1. Sum 1800,4 (with decimal comma on a German system). The values are strings; the conversion turns them into numbers.
  2. A list of objects is two levels deep; still give -Depth (a value of 3 or more) so that later changes cut nothing off.
  3. ($json | ConvertFrom-Json)[0].reise gives Oslo.

(Afterwards: [IO.Directory]::Delete($root, $true) cleans up the practice folder.)

What you can do now

  • You use providers and drives (Env:, file system) with the same commands.
  • You give absolute paths in .NET calls and take -LiteralPath for names from data.
  • You choose the encoding of files explicitly and know the difference between PowerShell 7 and Windows PowerShell 5.1.
  • You read and write CSV (values are strings) and JSON (always with -Depth).
  • You search with -match and Select-String and call web interfaces with Invoke-RestMethod.

This ends the first module. In the second module you write your own scripts.

Further reading