Programming

YouTube Video Metadata Scraping with PowerShell

FoxDeploy2 Comments

Trigger Warning : I discuss eating disorders and my opinions pro-eating disorder media briefly in this post. If this content is difficult for some, I recommend scrolling past The Background and resuming at The Project instead.

Background

I ❤ YouTube. I have learned so much about development from folks like I am Tim Curry, or from the amazing Microsoft Virtual Academy courses from Jeffrey Snover and Jason Helmick (original link ).

Most days I catch the repeats from Stephen Colbert, and then jam out to synthwave or chillhop. In fact, I listened to one particular mix so many times while learning c# that I still get flashbacks when I hear the songs on it again…sleepness nights trying to uncover everything I don’t know. I even have my own Intro to PowerShell Video that I think my mom watched 70,000 times.

My kids grew up singing songs from Dave and Eva, Little Baby Bum, Super Simple Songs and now Rachel and the TreeSchoolers, and it was one of the first services I signed up for and still pay for today (aside from NetFlix, and that one stint where I got CDs through the mail, yeah…)

But a few months ago I heard that YouTube will recommend videos which are pro eating-restriction and bulimia within four videos of the sorts of content targeted at young children. I have a history with people who experience these disorders and want to be sure we face it head on in my family, but that doesn’t mean I will allow impressionable minds to be exposed to content which presents this issue in a positive light.

If YouTube is not going to be safe for the type of stuff my children want to watch, I needed to know.  Unfortunately the person who told me of this can not remember their source, nor could I find any decent articles on the topic, but I thought that this smelled like a project in the making. Continue reading →

Progressive Automation: Part I

FoxDeployLeave a comment

Progressive automation - real world automation in increasing complexity

In this series, I thought it’d be fun to walk through the common phases of an automation initiative and specifically show how I love to handle this sort of situation when it arises today.

We’ll walk through recognizing a good opportunity to move a manual task to automation covering these three main steps, over the next few posts here:

  • Begin with something terrible and manual and ease the pain by adding a simple script
  • Increase the sophistication and take it to the next level by adding a User Interface
  • Migrate our Automation from a PowerShell UI to a simple and easy asp.net portal which calls a script to run the task

Depending on the amount of steam I have left, we may even go one step further and make our dotnet site more advanced, if you all are interested ☺

Our goal is to go from ‘hey it actually worked’ to ‘It works pretty well now’, to ‘hey it actually still works!’

Tell me where it hurts

You should always start your automation by finding the biggest pain points or wastes of time and starting there.  Ideal cases are things that:

  • Require your specific manual intervention (+3 points)
  • Have to happen in an off hour or over the weekend (+5 points)
  • Are hard to do, or repetitive  (+5 points)
  • Have a nasty penalty if you get them wrong (+5 points)

Add them up and if you’re over 10 then you should think about automating it. Hell, if it’s over 6, you should automate it. Continue reading →

PowerShell – Testing endpoints that perform Anti-forgery verification

FoxDeployLeave a comment

First off, big thanks go to 🐦Ryan Ephgrave, an incredibly talented and easy to work with PowerShell and dotnet god I have the pleasure to learn from over at #BigBank™ (its a great thing LinkedIn doesn’t exist…)

We had a situation arise recently where we needed to create some Integration tests in Pester to validate a long list of web pages to be sure they responded after a deployment.  I started out manually writing a litany of Pester tests by hand like this:


Context 'Post Deployment Validation' {
    It 'Website #1 should be accessible' {
        $url = 'https://someserver:someport/someEndpoint'
        $results = Invoke-WebRequest -Uri $url -UseDefaultCredentials
        $results.StatusCode | should be 200
    }

    It 'Website #2 should be accessible' {
        $url = 'https://someOtherserver:someport/someEndpoint'
        $results = Invoke-WebRequest -Uri $url -UseDefaultCredentials
        $results.StatusCode | should be 200
    }[...]
}

I spoke with the team about what I was doing and Ryan drew my attention to the very neat TestCases of Pester, which you can read more about here.

With a bit of work, I converted my long list of tests (which I typed by hand…why?  Because I finally got a PS4 and I stayed up too late playing Sekiro!) into a JSON file like this.

[
    {
        "SiteName" : "Our Home Page",
        "Url" : "https://someserver:someport/someEndpoint"        
    },
    {
        "SiteName" : "Our WebApp #1",
        "Url" : "https://someOtherserver:someport/someEndpoint"        
    }
]

Then to hook this up to our Pester test from before and… Continue reading →

Life after Write-Debug

FoxDeploy8 Comments

Hey y’all.  I’ve been getting verrrry deep into the world of Asp.net Model View Controller and working on some big updates to ClientFaux, but I saw this tweet and it spoke to me:

Why?  Because until recently, I was notorious for leaving Write-Debug statements everywhere.  I mean, just take a look at my local git folder.

A PowerShell console window running the following command. Dir c:\git -recurse | select-string 'write-debug' | measure This shows that there are over 150 uses of this command in my PowerShell modules. Uh, probably too many!
I *wasn’t* expecting it to be *this* bad. I’m so, so sorry.

My code was just littered with these after practically every logical operation…just in case I needed to pause my code here at some point in the future.  Actually, someone could look at my code in the past and every Verbose or Debug cmd was basically a place that I got stuck while writing that cmdlet or script.  I mean, using the tools is not wrong, but it always felt like there should be better ways to do it.

Recently, I have learned of a much better way and I want to share it with everybody.

Why not use Write-Debug?

Write-Debug is wrong and if you use it you should feel bad

I’m just kidding!  You know, to be honest, something really gets under my skin about those super preachy posts like you always find on medium that say things like ‘You’re using strings wrong’, or “You’re all morons for not using WINS” or something snarky like that.

It’s like, I might have agreed with them or found the info useful, but the delivery is so irksome that I am forced to wage war against them by means of a passive aggressive campaign of refusing to like their Tweets any more as my retribution.

That being said, here’s why I think we should avoid Write-Debug.  It ain’t wrong, but you might like the alternative better.

Continue reading →

Excursion: Model View Controller Programming – Part I

FoxDeployLeave a comment

The header graphic, titled - Excursion Model View Controllers. Subtitle: getting side-tracked in the land of dotnet. Pictures a single adventurer from far away, bundled up against the cold, trekking up the side of a snowy mountain

Well, it’s been a minute, hasn’t it?  This 🦊 needed a break after speaking at MMS and PSChatt! (both awesome events!  If you’re shopping for a big conference, I can’t recommend #MMSMOA enough!  And early bird pricing is open now!).

Since then at work I’ve had to dive deep and learn a whole new skill set.

Fortunately, I had a scuba tank and now that I’m back up for air, I’d love to share a bit of what I learned.

This is kind of different

It’s been a long term goal of mine to expand beyond my PowerShell capabilities and learn as much as I can of this ‘programmer’ stuff. I’d always wanted to learn C#, and my first deliverable was the ‘at least kind of working’ Client Faux (big updates to come).

Our goal was to make a cool website where end users could go and type in manually, or provide a CSV file of devices, and I’d use that to spin up new collections with deployments and then perform some automation on those devices within SCCM.  I want to talk through how I’m doing that, and the goal of this post should lay a foundation to answer the question of: what is a model view controller(mvc)?  Spoilers, MVCs are all around us!

So to recap our goal:  I needed to have a user give me input and a csv, then parse and slice it and show them the results to edit or tweak. That’s going to be our end game for this guide.
But before we talk about the technology…

But Stephen, are you qualified to teach me about this?

Uhhhh…maybe. I may not have all of the terminology down pat, and there might be a more efficient way of doing things than I have figured out.  But, ya know, I did just figure this out, plus I’m willing to share with you, so what else are you gonna do? 😁🦊

The technology stack

The goal was to host the site in IIS, with an ASP.Net Model View Controller and the powerful Entity Framework to handle my DB interactions. To throw some jargon, an ASP.net MVC with EF 6. Continue reading →