Extracting and monitoring web content with PowerShell

 

Extract PowerShellThis kind of request comes up all the time on StackOverflow and /r/PowerShell.  “How can I extract content from a webpage using PowerShell”.

This post COULD have been called ‘Finding a Nintendo Switch with PowerShell’, in fact!  I have been REALLY wanting a Nintendo Switch, and since I’ll be flying up to NYC next month for Tome’s NYC TechStravaganza (come see me if you’ll be in Manhattan that day!), it’s the perfect justification for She-Who-Holds-The-Wallet for me to get one!

But EVERYWHERE is sold out.  Still!  😦

However, the stores have been receiving inventory every now and then, and I know that when GameStop has it in stock, I want to buy it from them!

The first step then is to find a page that has the information I want to extract. Continue reading

Advertisements

Advanced Autocompletion: adding output types

upgrade-your-code

This post is part of the series on AutoCompletion options for PowerShell! Click the banner for more posts in the series!


Previously in this series, we reviewed a few ways to add AutoComplete onto your functions, covering Param AutoCompletion and Dynamic Parameters.  In this post, we’ll spend a LOT of time typing in the present to help our future selves save fractions of a second, because there’s no way we’ll become less lazy, right?  At the end of the day, we will have achieved the holy grail of Attaboys, and have Output Autocomplete working in our function.

Output AutoComplete

You know how in PowerShell you can type a cmdlet, then pipe into Select-Object or another cmdlet and start tabbing through property names?  This is the type of Autocompletion we are going to add to our function in this post!

gif

Not only does this save you from making mistakes, but it is amazingly convenient and really gives our functions a polished and professional look and feel.  PowerShell’s ability to do this highlights one of its distinguishing features as well!

Dynamic Type System

Warning: this next part is probably kind of boring

If you’re like me, you read things and then just gloss over all of the words and symbols you don’t know, assuming they’re unimportant.  If I just described you, then I hate to be the one to tell you this, but that is kind of a tremendous character flaw.  I’ll get around to why this is bad and how it relates to PowerShell, but first, take us on a detour into my past. Continue reading

Lessons on ProcMon and how to force On-screen keyboard

No On Screen Keyboard-

Recently, I had a customer looking at setting up potentially tens of thousands of Point of Sale Kiosks running Windows 10 on an LTSB branch.  We wanted users to have to input their password, but noticed that if a Windows 10 machine is in the docking station, the Touch Keyboard will never display!

Paradoxically, if the user has a Windows Hello Pin specified, that version of the touch keyboard will appear. But for a regular password?  Nope, no On-Screen Keyboard.  And using the dated compatibility keyboard (OSK.exe) was not an option.

To illustrate how weird this confluence of conditions was, I’ve provided a video

While we wait for Microsoft to create a patch to fix this, I’ve created my own workaround, using WMI Events and PowerShell!

In a perfect world, we’d wait for a hotfix. If it affected many people, Microsoft would roll out a patch for it.

Life isn’t perfect and we don’t have time to wait!  Sometimes all you really need is to open up Process Monitor and then write your own hack.

Continue reading

Is WinRM Secure or do I need HTTPs?

One of the things I absolutely love about my job is being thrown into the deep end of the rapids with little to no time to prepare  given the opportunity to try new things and new technologies, pushing me out of my comfort zone.  It normally goes okay.

whitewater
actual camera footage of my last project

Case in point: a client of ours recently was investigating WinRM and whether or not it was secure, leading me down a rabbit hole of Certificates, Enterprise CA’s, SSL Handshakes, WireShark and more.

At the end of the initiative, I was asked to write up a summary to answer the question

Is WinRM secure or do I really need HTTPs too

In this post, I’ll talk us through my findings after days of research and testing, stepping through the default settings and some edge cases, hopefully covering the minimum you need to know in a short little post.

Authentication Security

Consider the following scenario: two computers, both members of the same domain.  We run winrm quickconfig on both computers and don’t take any additional steps to lock things down.  Is it secure?  Are credentials or results passed in the clear?  Until stated otherwise, assume HTTP until I mention it again.

From the very first communications and with no additional configuration, connections between the two computers will use Kerberos for initial authentication.  If you’re not familiar with it, the bare minimum to know is that Kerberos is a trusted mechanism which ensures that credentials are strongly protected, and has a lot of nifty features like hashing and tickets which are used to ensure that raw credentials never go over the wire.  So, domain joined computers do not pass creds in the clear. Continue reading

SOLVED: What happens to WINRM when certs die

the-case-of-the-ghost-certificate-p2

Oh boy, this has been a rollercoaster of emotions.  But guys…we made it.  We have finally, and definitively answered what happens to WinRM with HTTPs when certificates expire.  If you’re curious about why this is a big question, see my previous posts on this topic.

Up until now, I’ve been able to say, conclusively, that WinRM generally seems to work, even as Certs expire and are renewed.  But I’ve never known why: did WinRM automatically update the certs?  Does Windows just not care about certs?  What is the purpose of life?

Well, I can now shed light on at least some of those questions.  I knew what I needed to do

Record a WireShark transfer and extract the certificate to tell definitively, which cert is being used to validate the session.  Then we’ll know what happens.

Setting the stage

Two VMs, one domain.  Server 2016 server, connected to from a Server 2012 R2 client. Newly created WinRM capable Certificate Template available to all domain members with a 4 hour expiration and 2 hour renewal period.

00-cert-temp

With the stage set, and the cert was present on both machines, I ran winrm quickconfig -transport:https on each, then made sure they could see each other, and remoted from one into the other.  I recorded a WireShark trace of the remote session, uh remoting, then ran a command or two, then stopped recording.  Then I opened the trace.

Continue reading