YouTube Video Metadata Scraping with PowerShell

Trigger Warning : I discuss eating disorders and my opinions pro-eating disorder media briefly in this post. If this content is difficult for some, I recommend scrolling past The Background and resuming at The Project instead.

Background

I ❤ YouTube. I have learned so much about development from folks like I am Tim Curry, or from the amazing Microsoft Virtual Academy courses from Jeffrey Snover and Jason Helmick (original link ).

Most days I catch the repeats from Stephen Colbert, and then jam out to synthwave or chillhop. In fact, I listened to one particular mix so many times while learning c# that I still get flashbacks when I hear the songs on it again…sleepness nights trying to uncover everything I don’t know. I even have my own Intro to PowerShell Video that I think my mom watched 70,000 times.

My kids grew up singing songs from Dave and Eva, Little Baby Bum, Super Simple Songs and now Rachel and the TreeSchoolers, and it was one of the first services I signed up for and still pay for today (aside from NetFlix, and that one stint where I got CDs through the mail, yeah…)

But a few months ago I heard that YouTube will recommend videos which are pro eating-restriction and bulimia within four videos of the sorts of content targeted at young children. I have a history with people who experience these disorders and want to be sure we face it head on in my family, but that doesn’t mean I will allow impressionable minds to be exposed to content which presents this issue in a positive light.

If YouTube is not going to be safe for the type of stuff my children want to watch, I needed to know.  Unfortunately the person who told me of this can not remember their source, nor could I find any decent articles on the topic, but I thought that this smelled like a project in the making. Continue reading

Progressive Automation: Part I

Progressive automation - real world automation in increasing complexity

In this series, I thought it’d be fun to walk through the common phases of an automation initiative and specifically show how I love to handle this sort of situation when it arises today.

We’ll walk through recognizing a good opportunity to move a manual task to automation covering these three main steps, over the next few posts here:

  • Begin with something terrible and manual and ease the pain by adding a simple script
  • Increase the sophistication and take it to the next level by adding a User Interface
  • Migrate our Automation from a PowerShell UI to a simple and easy asp.net portal which calls a script to run the task

Depending on the amount of steam I have left, we may even go one step further and make our dotnet site more advanced, if you all are interested ☺

Our goal is to go from ‘hey it actually worked’ to ‘It works pretty well now’, to ‘hey it actually still works!’

Tell me where it hurts

You should always start your automation by finding the biggest pain points or wastes of time and starting there.  Ideal cases are things that:

  • Require your specific manual intervention (+3 points)
  • Have to happen in an off hour or over the weekend (+5 points)
  • Are hard to do, or repetitive  (+5 points)
  • Have a nasty penalty if you get them wrong (+5 points)

Add them up and if you’re over 10 then you should think about automating it. Hell, if it’s over 6, you should automate it. Continue reading

PowerShell – Testing endpoints that perform Anti-forgery verification

First off, big thanks go to 🐦Ryan Ephgrave, an incredibly talented and easy to work with PowerShell and dotnet god I have the pleasure to learn from over at #BigBank™ (its a great thing LinkedIn doesn’t exist…)

We had a situation arise recently where we needed to create some Integration tests in Pester to validate a long list of web pages to be sure they responded after a deployment.  I started out manually writing a litany of Pester tests by hand like this:


Context 'Post Deployment Validation' {
    It 'Website #1 should be accessible' {
        $url = 'https://someserver:someport/someEndpoint'
        $results = Invoke-WebRequest -Uri $url -UseDefaultCredentials
        $results.StatusCode | should be 200
    }

    It 'Website #2 should be accessible' {
        $url = 'https://someOtherserver:someport/someEndpoint'
        $results = Invoke-WebRequest -Uri $url -UseDefaultCredentials
        $results.StatusCode | should be 200
    }[...]
}

I spoke with the team about what I was doing and Ryan drew my attention to the very neat TestCases of Pester, which you can read more about here.

With a bit of work, I converted my long list of tests (which I typed by hand…why?  Because I finally got a PS4 and I stayed up too late playing Sekiro!) into a JSON file like this.

[
    {
        "SiteName" : "Our Home Page",
        "Url" : "https://someserver:someport/someEndpoint"        
    },
    {
        "SiteName" : "Our WebApp #1",
        "Url" : "https://someOtherserver:someport/someEndpoint"        
    }
]

Then to hook this up to our Pester test from before and… Continue reading

ClientFaux 2.0 – Completely re-written, faster than ever

As mentioned on the stage at MMSMOA, ClientFaux 2.o is now available.  Completely re-written with as a WPF GUI with automated certificate generation, multi-threading, and all the bells and whistles.

Oh, and Hardware inventory now works!

Download it and give it a try now!  To use, install it on a desktop/laptop/VM which is on a network segment which can reach your CM server.

http://bit.ly/ClientFaux

Launch ClientFaux and click to the Configure CM tab and provide your CM Server FQDN and three letter Site code.

Then click to the Device Naming page and provide your desired naming pattern and starting and ending numbers.

You can also increase the number of threads (I’ve tested up to 12 threads and seven is a good happy medium for resource usage, but feel free to go crazy).

Then to see it in action…click to the ‘Ready’ page and hit ‘Ready!’ and away we go!

 

The Big Warning

This is designed for DEMO or TestLab CM instances.  I do not recommend running it against your Production CM instance as it can create thousands and thousands of CM clients if left running for a few hours!  This can be hard to filter out of data for reporting, dashboards and the like.

Quickie: ConvertTo-PSCustomObject

Do you ever need to quickly hop between PowerShell tabs in VScode, or have data you want to move from one session to another?

Sure, you could output your data into a .CSV file, a .JSon file, or one of hundreds of other options.  But sometimes it’s nice to just paste right into a new window and get up and running again.  For that, I wrote this small little cmdlet.

 Function ConvertTo-PSCustomObject{
    Param($InputObject)
    $out = "[PSCustomObject]@{`n"
    $Properties = $InputObject | Get-Member | Where MemberType -eq Property
    ForEach ($prop in $Properties){
        $name = $prop.Name
        if ([String]::IsNullOrEmpty($InputObject.$name)){
            $value = $null
        }
        else {
            $value = $InputObject.$name
        }

        $out += "`t$name = '$value'`n"
    }

    $out += "}"
    $out
}

And the usage of it:

ConvertTo-PSCustomObject