Troubleshooting PowerShell and .NET: When error messages are not enough.

Mar 5, 01:18 PM

Recently we had a report in the dbatools project: there was an issue installing from the PowerShell Gallery (via Install-Module.)

Error “Invalid characters in path.”

We had a few scattered notifications of the issue and even one of the dbatools developers affected internally, but it proved to be a stubborn bug and the people involved had tried numerous methods in tracking it down.

We knew that one of the paths being provided to Windows was bad as a part of our module was bad, but the error message sucked and everything worked almost everywhere; we were really confused about what was going on.

It was clear that like any new issue it was a product of new code and changes to the project, but it was unclear which of the hundreds and hundreds of new additions was the source were the culprit.

We had been adding new things at a frenetic pace and this was not something that came up during the standard build process… the final count was 185+ files which might have something to do with this error.

I finally was able to get the source of the problem through attaching my new favorite program dnSpy to my PowerShell process, and I wanted to show you how to do it too.

Process

First, you need to download and extract dnSpy :

  • Launch dnspy.exe – you may have to run dnSpy as an administrator to attach to processes or rewrite DLLs, I always launch it as administrator.
  • Switch to a ready PowerShell window with your command
  • From the debug menu, choose “Attach to Process…”
  • Find your powershell.exe in the list, and choose “Attach”
  • Back in your PowerShell window, run your command, you should see a new colored bar on the botton of the dnSpy window indicating the attachment worked and it is running.
  • Your PowerShell command should fail, and dnSpy should throw the exception and wait for input.
  • Click the “Call Stack” tab
  • Right Click in the Call Stack area and make sure “Show Parameter Values” is checked.
  • Success!!

You should now be able to read the methods that were called to get you to this error, and hopefully be able to suss out which methods got which parameters!

Further reading for dbatools problems:

When the module loaded and was analyzed, one of our strings in our psm1 file was interpreted as a filepath (but only in WMF 5.0, as stated 5.1 does not have this issue.)

The team was interested in keeping the module installation and setup process as speedy as possible and in that vein the following tweet and blog post came up

From the article:

Solution? There are few options (like pipe Get-Content to Invoke-Expression), but I ended up with dot-sourcing a script block created from content of the script.

It looks like while this was (and probably is) a great idea, in certain versions of Install-Module it will actually interpret the dot in the dot sourcing as the start of a path, and will attempt to evaluate it as such, throwing an absolute tizz.

In the mean time, dbatools has made changes to effectively do the same thing without using a dot prefix for the line (even though WMF 5.1 is not affected by this issue) and thankfully with a few other changes one of our core contributors Fred was able to find a fix without losing too much time.

Constantine Kokkinos
---

TIL - Trello(JSON) and PowerShell - Summarizing public boards

Jan 22, 09:18 AM

Had a need to look into a Trello board's activity and thought I would blog the first of some "recipes" I am coming up with.

To start it off, I wrote some trivial functions to format Trello URLs (as they appear today) to return the JSON representation of the board, and return some basic information about the activity that recently occurred.

This Gist is not a full fledged module as of yet, just a bag of functions and a little extra code. On line 54 I define a hash of boards and then break out the activity and cards updated in the last 10 days.

More than anything this code had me diving deeper with JSON, spending some time figuring out what I could do with PowerShell Format Expressions, and a bit of checking/capturing the implicit type conversions PowerShell loves to throw at me.

I know there is more to do here and I could be significantly more terse, could remove the hash table, and the naming might not work if they provide a url without the pretty printing, so I could definitely add more sanity checking in the Format-TrelloUriToJson function.

Constantine Kokkinos
---

T-SQL Tuesday 85: Managing database snapshots with dbatools

Dec 13, 07:41 AM


For my second TSQL Tuesday post, I wanted to introduce a nifty feature in the dbatools lineup called Restore-SQLBackupFromDirectory, which used Ola Hallengren’s scripts as a baseline to restore a server from a backup folder, un/fortunately I came across a newer command that Stuart Moore engineered (Restore-DBABackup) which made it entirely redundant, expect his post coming very soon.

Since there was a bit of a last minute pivot, we are going to do a whirlwind tour of two new PowerShell commands that just hit the lineup regarding database backups, Get and Remove DbaDatabaseSnapshot.

These along with an additional function coming soon (New-DbaDatabaseSnapshot) are going to make it super simple to create, list, and remove database snapshots on your SQL Server instance.

Command Reference

Listing snapshots on a target SQL Server

Get-DbaDatabaseSnapshot -SqlServer [-Credential ][-Databases ] [-Exclude ]

Returns a list of database snapshots on your SQL Server.
Accepts multiple SQL Servers, and can filter inclusively or exclusively on the base database name to find snapshots.

Example

Removing snapshots on a target SQL Server

Remove-DbaDatabaseSnapshot -SqlServer [-Credential ] [-Snapshots ] [-Databases ] [-Exclude ]

Drops snapshots from your SQL Server.
This has an additional unique parameter of -Snapshots, which allows you to filter on exact snapshot name instead of using a base database name (in the case of multiple snapshots you want to remove.)

Example

Common Parameter Reference

As with many commands in the dbatools lineup there are several common parameters:

  • SqlServer – Indicates what server you would like to operate on.
  • Credential – Allows you to include an overriding SQL Login credential (PowerShell object),
  • Databases – Operates as an inclusion filter for database names.
  • Exclude – Operates as an exclusion filter for database names.

Final Notes:

Hopefully after reading today you have two commands ready to help you manage your database snapshots, and I have whet your appetite for the upcoming restore and snapshot commands coming soon!

  • If you don’t see the filtering/parameter populating from tab completion, this indicates an issue connecting to your SQL Server.
  • Both of these commands were written recently by one of the newest contributors to the project, niphlod, great job!
  • If you want to learn more about dbatools (free PowerShell module with nearly 100 SQL Server administration, best practice and migration commands included) checkout https://dbatools.io, we have a growing team and we want contributions from busy SQL DBAs like yourself.
  • If you are unsure where to start or just want to join the growing communtity, you can find us on the SQL Community Slack feel free to send me a message if you need some help!
Constantine Kokkinos
---

TIL - Nov 25 2016, xp_dirtree appends null bytes to filenames in SQL 2000

Nov 25, 09:13 AM

I was performing some final minor fixes on Find-DbaOrphanFile (helps cleanup files unattached to your server in default directories and additional folders you specify) and ran into an issue parsing the filenames returned.

In the process of changing the code to fix some other issues, IO.Path.GetFileName was added to properly enumerate the paths returned, and when testing it in the dbatools lab (Thanks @ctrlb!!!) it was failing on SQL 2000.

After a few attempts at parsing out slashes and various .trim commands I broke out the command in the PowerShell debugger (which I will detail in another post) and I was thinking of converting everything to numeric representations to double check their character values, but there was no need, the console had done the work for me:

The Find-DbaOrphanedFile (now with an additonal ltrim and rtrim in SQL instead) command now works!

As a little aside, why even test SQL 2000?

One of the first things dbatools strove to do was migrate effectively between different versions of SQL Server, and we want to support SQL 2000 up to the most recent SQL version if possible, we do this because we want to offer the cool tools as far back as possible, we know that many people will be using old versions for many years to come.

It hurts sometimes and some of the code can get a bit hacky, but I think most of SQL 2000’s differences are interesting and long term forcing ourselves to manage different versions of SQL Server is reducing the into adding new and fundamentally different databases, such as all the interesting things that are coming with vNext.

And as always, this is one of the many dangers of using undocumented stored procedures :)

Constantine Kokkinos
---

T-SQL Tuesday 84: Exploring DBA Tools or: How I Learned to Stop Worrying and Love PowerShell

Nov 8, 08:00 AM

@sqlbek (Andy Yun) posted a challenge for T-SQL Tuesday 84, where he encouraged people thinking about becoming a speaker to write about their experience giving their first talk, a topic they wanted to present on in the future, or even on starting the road to creating their blog and establishing themselves in the SQL space.

I wanted to flesh out an idea I had for a my first talk that would last under 15 minutes or so.
I was thinking of targeting this at a user group (hopefully with a few production DBAs where are interested in learning more about PowerShell) to foment some excitement for and share my experience with dbatools.io

who (am I)

  • I am a production DBA who focuses on performance tuning for a few thousand users.
  • I haven’t written anything you’ve read (yet).
  • I have been poking around with PowerShell for more than a year, before this experience I had written only a few small things that made it to production. Definitely beginner skill level.

have you ever wanted to…

  • Migrate your SQL Server painlessly?
  • Double check that you have the best practices configuration for component or feature X which you are not an expert in?
  • Centralize the knowledge you find scattered around on various blogs and SQL wizard cabals?

what (do I want to tell you)

I am here to introduce your to dbatools, an open source project, to talk about my experience going from pure production DBA to PowerShell convert, and to talk to you about how augmenting your skills with some of these commands can mean very little effort expended for a significant payoff.

As a production DBA, I am responsible for a host of things, and I am often a little buried under the mountain of work that piles up, along with every day work, among other things I am responsible for:

  • Backups
  • Security
  • Managing performance
  • General Administration
  • Reporting on and managing configuration drift

My environment is growing rapidly, so I started looking for tools that I could use to help me do my job in a way that didn’t leave me in a cold sweat at night.

enter dbatools

dbatools was created help busy DBAs do their jobs, hopefully encoding best practices and reducing the cargo culting in the SQL Server community.

While dbatools was originally designed to migrate your SQL Server instance’s objects from one server to another it is currently expanding into a set of commands (cmdlet reference ) that I think many of my peers in the SQL Server would find very useful.

I first heard about dbatools in the SQL Community Slack and loved the demo on the front page showing how simple a migration is:

I was a bit gobsmacked as I had just finished my own instance migration, and it certainly took a significant amount of time and effort compared to this painless invoking of this cmdlet.

A bit of hanging around and even though I felt unfamiliar with PowerShell development, I was was roped into writing my own cmdlet .

To be frank, looking back at my first draft is painful.

I was worried about my code quality given my experience, and so I asked for some feedback from the head of the project Chrissy LeMaire.

What I got back was almost unrecognizable as my code, and it was one of the first times I was able to see my feral PowerShell (that did the job, badly) turned into something that you could consider idiomatic code.

I learned more in that code review than I had in the previous year of starts and stops in PowerShell land and spent the next week picking it apart and asking questions to cement my knowledge.

In one week I had expanded my PowerShell knowledge by at least 20 times, and felt way more comfortable in working with the team to build some command or troubleshoot an issue!
In the end I had a fairly robust cmdlet and with minor bug fixes it made it into the release!
I was super happy to have something published in an open source project, but even more, I was happy to have found a series of mentors who wanted to help me build my skills as a PowerShell and database developer.

A few pull requests and a few months later I am confident in my PowerShell abilities to figure stuff out, work through problems, and help newbies through their first steps to contributing to dbatools.

The small scope of PowerShell I have been able to learn and utilize has made things which at one time seemed daunting now seem possible, and even easy.

Now I want to help you make my story your story, contribute to dbatools.

when\when

Any time you want to.

dbatools is a community that lives in the #dbatools channel in the SQL Community Slack.
Any time day or night people are chatting, working on code, listening on feedback to improve the tools, and answering questions for people looking for help using it.

Come and express interest in contributing, we will help you through the rest!

how (do you contribute)

  • Join the #dbatools channel in the SQL Community Slack
  • If you are just learning PowerShell, that’s ok, many of us are still puzzling things out! All skill levels welcome!
  • We need help with not just with writing code, but with documentation and testing; your first contribution could be as simple as running a command.
  • You can contribute as much or as little as you have time for, many issues are as simple to fix as adjusting one line of code.
Constantine Kokkinos
---

« Older Newer »