86 lines
9.1 KiB
Markdown
86 lines
9.1 KiB
Markdown
# File download diagnosis
|
|
|
|
`Test-FileDownload.ps1` compares the loaded `Download-File` helper, synchronous BITS, an inspectable BITS job, and a direct .NET GET. It makes no changes to `Tools.ps1`, collector behavior, Windows TLS configuration, or proxy configuration. Downloaded files are never executed.
|
|
|
|
## Run on the affected host
|
|
|
|
Use the same RMM account, PowerShell executable, and bootstrap as the failing job. Keep the existing Tools bootstrap unchanged and replace the final collector invocation with this block after publishing the diagnostic script to the repository:
|
|
|
|
```powershell
|
|
$DiagnosticClient = New-Object Net.WebClient
|
|
try {
|
|
$DiagnosticText = $DiagnosticClient.DownloadString(
|
|
'https://dev.emberkom.com/emberkom/management-scripts/raw/branch/master/Test-FileDownload.ps1'
|
|
)
|
|
}
|
|
finally { $DiagnosticClient.Dispose() }
|
|
|
|
& ([scriptblock]::Create($DiagnosticText)) `
|
|
-URL 'https://xfer.emberkom.com/shares/tools/folders/9b9988fb-c53a-4833-a4ad-68d01a3021bb/files/dsa-collect-windows-amd64-exe/download/dsa-collect-windows-amd64.exe' `
|
|
-OutputDirectory "$Env:ProgramData\Emberkom\Output" `
|
|
-TimeoutSeconds 120
|
|
```
|
|
|
|
Do not invoke it through `Download-File`, since that is the function being investigated. Alternatively, save the diagnostic on the affected host and call it directly with `& 'C:\path\Test-FileDownload.ps1' -URL '<download URL>'`. If Tools cannot load on a legacy engine, a standalone run still tests the other methods and records that the helper was unavailable.
|
|
|
|
Use `powershell.exe`, not ISE or an embedded PowerShell host. The diagnostic rejects other executables instead of guessing how to launch equivalent child processes.
|
|
|
|
The mandatory `-URL` can be any HTTP(S) file URL without embedded credentials. `-OutputDirectory` defaults to the account's temporary directory. Each invocation creates its own `file-download-diagnostic-<GUID>` subdirectory, containing `report.txt` and `report.clixml`. No reports are uploaded automatically. Retrieve `report.txt` from the affected host for analysis; the CLIXML file preserves structured details.
|
|
|
|
There are ten sequential probes: environment, HTTP headers/ranges, and two destination variants for each of four download paths. Eight probes download the payload, so use a small representative file. Each probe has a 120-second default wall-clock limit, followed by a separate cleanup attempt of at most 15 seconds for BITS-related probes. Allow about 22 minutes plus process startup/reporting overhead for the worst case, or specify a shorter timeout. Large downloads and hashing count toward the timeout.
|
|
|
|
## What the report captures
|
|
|
|
- OS, CLR and PowerShell versions, process bitness, executable, identity, BITS service/module information, and proxy/TLS settings. Missing registry values are recorded rather than invented as defaults.
|
|
- Definitions and SHA-256 hashes of the loaded helper functions, plus command resolution. The full Tools script is not executed by the diagnostic.
|
|
- HTTP redirect chains, status, content length, range support, and selected response headers. Cookies and authorization headers are not collected.
|
|
- Independent downloads to new destinations and existing empty destinations, byte counts, hashes, first bytes, timing, and full exceptions.
|
|
- BITS job state changes and byte counts before completion, and file snapshots immediately after transfer and one second later.
|
|
|
|
Child processes use the caller's PowerShell executable and account, with its .NET TLS selection reproduced in the child only. Existing pooled connections, custom in-memory proxy objects, and certificate callbacks are not copied. The helper probe copies the captured functions, redirects helper logging into diagnostic scratch, and adds ownership tags to BITS jobs. A shadowed/non-native `Start-BitsTransfer` is recorded and the helper probe is skipped rather than risking cleanup of unidentified jobs.
|
|
|
|
Scratch filenames retain the original URL's extension when usable, but the production destination is never touched. Path-specific security rules may therefore behave differently. The loaded helper's logging overrides are copied only into its child process.
|
|
|
|
Only jobs matching both the diagnostic's unique run ID and exact probe tag are removed. Scratch cleanup does not recurse or touch the production executable. Cleanup errors and unfinished probes remain visible in the report. Reports include local machine/account details and loaded source definitions; review them before sharing.
|
|
|
|
The code uses PowerShell 2 syntax and APIs available to legacy .NET, but local validation on PowerShell 5.1 does not prove execution on 2, 3, or 4. Run the diagnostic on actual legacy runtimes before declaring them supported. Full `Tools.ps1` loadability is a separate issue: it already contains newer syntax and commands, including `-in`/`-notin`, `::new()`, `ConvertFrom-Json`, and `Get-FileHash` outside the downloader.
|
|
|
|
## Interpret the comparison
|
|
|
|
| Observation | Next investigation |
|
|
|---|---|
|
|
| Only the helper fails | Compare `HelperBitsSource` with the original URL, the redirect chain, and URL-resolution requests. |
|
|
| BITS fails and direct GET succeeds | Compare service identity, WinHTTP/.NET proxy and TLS settings, HTTP behavior, and BITS error codes. |
|
|
| Both transports fail | Investigate endpoint responses, certificate trust, connectivity, and the affected machine's configuration. |
|
|
| BITS reports bytes transferred but the resulting file differs | Inspect completion errors, destination access, and post-download changes. |
|
|
| All paths produce the same nonempty hash | Failure was not reproduced. This run does not justify changing the shared transport. |
|
|
|
|
Neither a positive byte count nor matching hashes authenticates a publisher's executable. These are diagnostic comparisons, not a replacement for publisher-supplied integrity information.
|
|
|
|
## Shared caller audit
|
|
|
|
The audit found 24 active call sites across 20 files: 19 scripts plus five wrapper call sites in `Tools.ps1`. Commented-out calls and tests are excluded.
|
|
|
|
| Callers | Required behavior |
|
|
|---|---|
|
|
| `Run-Script`, `Source-PSScript`, `Uninstall-MicrosoftOffice` | Return one completed script path with its extension; allow immediate sourcing/execution. |
|
|
| `Install-MSI`, `Install-4KVideoDownloader`, `Install-LiquidFilesOutlookAgent`, `Install-SimpleInOut`, `Install-SpecsIntact`, `Install-Wireguard` | Return a completed installer path for immediate MSI invocation and later cleanup. |
|
|
| `Install-Nextcloud`, `Install-OpenVPN`, `Install-VLC` | Work inside a subexpression or via positional URL; return only the downloaded path. |
|
|
| `Install-AutodeskDesktopConnector`, `Install-Enscape`, `Install-ESETManagementAgent`, `Install-MicrosoftOffice365`, `Install-SketchUp` | Honor an explicit destination; handle redirects and potentially large installers; finish before installation. |
|
|
| `Fix-AutodeskProductLicensing`, `Test-InternetBandwidth`, `Get-LiquidFilesCLI` | Retain a usable ZIP filename and extension for extraction and derived working-directory names. |
|
|
| `Get-DSAManifest`, `Fix-UpdateLocalHosts`, `Remove-AllESETProducts`, `Install-LocalTools` | Honor fixed executable paths and complete replacement before execution. Caller-level caching remains separate. |
|
|
|
|
All concrete download URLs in this audit use HTTPS. The public wrappers also accept caller-provided HTTP(S) URLs. No audited caller consumes a BITS job object or explicitly controls background transfers/resume.
|
|
|
|
`Download-File` currently calls `Get-AbsoluteURI`, which first calls `IsURLValid`; both issue GET requests without disposing their responses. The final response URI is then passed to BITS. This is an observed implementation detail to investigate, not proof that BITS should be replaced.
|
|
|
|
Any subsequent shared fix must preserve URL binding, explicit/automatic filenames, synchronous completion, one success path on the output stream, redirects, large-file streaming, shell architectures, and `Run-Script` shared scope. It must also address validation and failure propagation before callers install or extract content. Production implementation and rollout follow the diagnostic findings; this change adds no fallback or collector-specific transport option.
|
|
|
|
## Local verification
|
|
|
|
Run `tests\Test-FileDownload.Tests.ps1` in both System32 and SysWOW64 Windows PowerShell. The suite serves synthetic payloads over loopback, tests both destination variants, redirects, chunked/empty/truncated/error responses, timeouts, hashing, and preservation of an unrelated BITS job. It requires access to the BITS service but performs no external downloads, installations, or uploads.
|
|
|
|
The affected RMM host and actual PowerShell 2-4 engines must still be tested separately. No local result substitutes for those environments.
|
|
|
|
Local verification on 2026-09-30 passed under both 32-bit and 64-bit PowerShell 5.1. A live 32-bit run against the AMD64 collector URL returned 3,455,488 bytes with SHA-256 `CC693EF2FEEEF05C99410B6F7A05AE2621EA0B89BA6D3E12C50B7F059D557F99` for every download method and destination variant. This is an observed comparison hash, not a publisher-provided trust anchor. The failure was not reproduced, so the current recommendation is to retain the shared transport until the affected RMM run supplies evidence.
|