— Constantine Kokkinos
I wanted to share a scenario and solution for a problem I am seeing arise in the dbatools project:
- Commits are registered against the “wrong” branch
- A pull request is submitted
- The file comparison shows hundreds of changes instead of 1-2
There are many ways this can happen, but usually due to forgetting to create a feature branch, or accidentally creating said branch from master instead of development.
To fix this problem, you will want check the log of commits to determine the hash of the command you want, and then create a new branch (from the correct source) and apply those commits.
git cherry-pick is a command that allows you to apply a commit on a different branch than what you what you originally intended.
To fix a pull request where you have more files than you want:
- Make sure you are in the branch you had a problem.
git status to see your current branch and
git checkout branchname will change your context to the branch name you want.
- Use the
git log command to see a list of commits, you should see your changes and a relevant hash.
- Record the first 10-12 characters of the commit hashes of your changes (technically you only need enough of the values to be unique.)
- Checkout the correct source branch (development) with
git checkout development
- Create the correct feature branch with
git checkout -b new-feature-name (this will also check it out for you and set it to be the current branch)
- Cherry pick the commits with
git cherry-pick hash
- Push the new branch to your repository
git push origin new-feature-name
- You should now be able to Pull Request/Compare your code.
A note on comparisons, any time you make a new Pull Request GitHub will automatically list the files changed and the number of commits you put in place, if this seems different than what you did, give me a jingle and we can see which part of this process we want to do :)
— Constantine Kokkinos
Recently we had a report in the dbatools project: there was an issue installing from the PowerShell Gallery (via Install-Module.)
Error “Invalid characters in path.”
We had a few scattered notifications of the issue and even one of the dbatools developers affected internally, but it proved to be a stubborn bug and the people involved had tried numerous methods in tracking it down.
We knew that one of the paths being provided to Windows was bad as a part of our module was bad, but the error message sucked and everything worked almost everywhere; we were really confused about what was going on.
It was clear that like any new issue it was a product of new code and changes to the project, but it was unclear which of the hundreds and hundreds of new additions was the source were the culprit.
We had been adding new things at a frenetic pace and this was not something that came up during the standard build process… the final count was 185+ files which might have something to do with this error.
I finally was able to get the source of the problem through attaching my new favorite program dnSpy to my PowerShell process, and I wanted to show you how to do it too.
First, you need to download and extract dnSpy :
- Launch dnspy.exe – you may have to run dnSpy as an administrator to attach to processes or rewrite DLLs, I always launch it as administrator.
- Switch to a ready PowerShell window with your command
- From the debug menu, choose “Attach to Process…”
- Find your powershell.exe in the list, and choose “Attach”
- Back in your PowerShell window, run your command, you should see a new colored bar on the botton of the dnSpy window indicating the attachment worked and it is running.
- Your PowerShell command should fail, and dnSpy should throw the exception and wait for input.
- Click the “Call Stack” tab
- Right Click in the Call Stack area and make sure “Show Parameter Values” is checked.
You should now be able to read the methods that were called to get you to this error, and hopefully be able to suss out which methods got which parameters!
Further reading for dbatools problems:
When the module loaded and was analyzed, one of our strings in our psm1 file was interpreted as a filepath (but only in WMF 5.0, as stated 5.1 does not have this issue.)
The team was interested in keeping the module installation and setup process as speedy as possible and in that vein the following tweet and blog post came up
From the article:
Solution? There are few options (like pipe Get-Content to Invoke-Expression), but I ended up with dot-sourcing a script block created from content of the script.
It looks like while this was (and probably is) a great idea, in certain versions of Install-Module it will actually interpret the dot in the dot sourcing as the start of a path, and will attempt to evaluate it as such, throwing an absolute tizz.
In the mean time, dbatools has made changes to effectively do the same thing without using a dot prefix for the line (even though WMF 5.1 is not affected by this issue) and thankfully with a few other changes one of our core contributors Fred was able to find a fix without losing too much time.
— Constantine Kokkinos
Had a need to look into a Trello board's activity and thought I would blog the first of some "recipes" I am coming up with.
To start it off, I wrote some trivial functions to format Trello URLs (as they appear today) to return the JSON representation of the board, and return some basic information about the activity that recently occurred.
This Gist is not a full fledged module as of yet, just a bag of functions and a little extra code. On line 54 I define a hash of boards and then break out the activity and cards updated in the last 10 days.
More than anything this code had me diving deeper with JSON, spending some time figuring out what I could do with PowerShell Format Expressions, and a bit of checking/capturing the implicit type conversions PowerShell loves to throw at me.
I know there is more to do here and I could be significantly more terse, could remove the hash table, and the naming might not work if they provide a url without the pretty printing, so I could definitely add more sanity checking in the Format-TrelloUriToJson function.
— Constantine Kokkinos
For my second TSQL Tuesday post, I wanted to introduce a nifty feature in the dbatools lineup called Restore-SQLBackupFromDirectory, which used Ola Hallengren’s scripts as a baseline to restore a server from a backup folder, un/fortunately I came across a newer command that Stuart Moore engineered (Restore-DBABackup) which made it entirely redundant, expect his post coming very soon.
Since there was a bit of a last minute pivot, we are going to do a whirlwind tour of two new PowerShell commands that just hit the lineup regarding database backups, Get and Remove DbaDatabaseSnapshot.
These along with an additional function coming soon (New-DbaDatabaseSnapshot) are going to make it super simple to create, list, and remove database snapshots on your SQL Server instance.
Listing snapshots on a target SQL Server
Get-DbaDatabaseSnapshot -SqlServer [-Credential ][-Databases ] [-Exclude ]
Returns a list of database snapshots on your SQL Server.
Accepts multiple SQL Servers, and can filter inclusively or exclusively on the base database name to find snapshots.
Removing snapshots on a target SQL Server
Remove-DbaDatabaseSnapshot -SqlServer [-Credential ] [-Snapshots ] [-Databases ] [-Exclude ]
Drops snapshots from your SQL Server.
This has an additional unique parameter of -Snapshots, which allows you to filter on exact snapshot name instead of using a base database name (in the case of multiple snapshots you want to remove.)
Common Parameter Reference
As with many commands in the dbatools lineup there are several common parameters:
- SqlServer – Indicates what server you would like to operate on.
- Credential – Allows you to include an overriding SQL Login credential (PowerShell object),
- Databases – Operates as an inclusion filter for database names.
- Exclude – Operates as an exclusion filter for database names.
Hopefully after reading today you have two commands ready to help you manage your database snapshots, and I have whet your appetite for the upcoming restore and snapshot commands coming soon!
- If you don’t see the filtering/parameter populating from tab completion, this indicates an issue connecting to your SQL Server.
- Both of these commands were written recently by one of the newest contributors to the project, niphlod, great job!
- If you want to learn more about dbatools (free PowerShell module with nearly 100 SQL Server administration, best practice and migration commands included) checkout https://dbatools.io, we have a growing team and we want contributions from busy SQL DBAs like yourself.
- If you are unsure where to start or just want to join the growing communtity, you can find us on the SQL Community Slack feel free to send me a message if you need some help!
— Constantine Kokkinos
I was performing some final minor fixes on Find-DbaOrphanFile (helps cleanup files unattached to your server in default directories and additional folders you specify) and ran into an issue parsing the filenames returned.
In the process of changing the code to fix some other issues, IO.Path.GetFileName was added to properly enumerate the paths returned, and when testing it in the dbatools lab (Thanks @ctrlb!!!) it was failing on SQL 2000.
After a few attempts at parsing out slashes and various .trim commands I broke out the command in the PowerShell debugger (which I will detail in another post) and I was thinking of converting everything to numeric representations to double check their character values, but there was no need, the console had done the work for me:
The Find-DbaOrphanedFile (now with an additonal ltrim and rtrim in SQL instead) command now works!
As a little aside, why even test SQL 2000?
One of the first things dbatools strove to do was migrate effectively between different versions of SQL Server, and we want to support SQL 2000 up to the most recent SQL version if possible, we do this because we want to offer the cool tools as far back as possible, we know that many people will be using old versions for many years to come.
It hurts sometimes and some of the code can get a bit hacky, but I think most of SQL 2000’s differences are interesting and long term forcing ourselves to manage different versions of SQL Server is reducing the into adding new and fundamentally different databases, such as all the interesting things that are coming with vNext.
And as always, this is one of the many dangers of using undocumented stored procedures :)