PowerShell was clogging up Tech For Non-Profits. For the past several weeks I wrote a bunch of posts about using Windows PowerShell to access web APIs. This has escalated to the extent that I thought it would worth an entire blog, so I’ve moved these posts to the brand-new shiny PowerShell Notebook. where I’m groping around for PowerShell mastery. So far I have code to show how to access the following APIs:
PowerShell: Use a Profile for ISE settings
The PowerShell profile is a .Ps1 startup script that runs when you first load the interactive script editor (ise). You can use it to load modules, set a default prompt and default directory, and to map abbreviations to longer command strings.
To see the location of the profile, you simply have to examine the $profile variable:
PS >$profile C:UsersLarryDocumentsWindowsPowerShellMicrosoft.PowerShellISE_profile.ps1
The profile can be opened like any other script.
To reload a profile after editing it, you simply need to invoke it using the ampersand “call operator”.
PS> & $profile
If you are reloading an already existing profile…then you may see lots of harmless errors as PS complains about settings that were already in place.
- Here’s my current profile. It does three things:
- Changes the prompt to PS>
- Changes to my default working directory
- Set an alias of gh for the get-help commandlet
- Set an alias of np to start notepad++
# Define the powershell Prompt.
function prompt
{
"PS>"
}
# Define the default working directory
Set-Location C:UsersLarryPowershell
PWD # show the current working directory
# Set an alias for the Get-Help command. Set-Alias gh Get-Help
# Set an alias to run the Notepad++ editor Set-Alias -Name np -Value "C:Program Files (x86)Notepad++notepad++.exe"
The last command took some experimentation, but I can now type np followed by a file name to open the notepad++ editor with that file.
Powershell: Use the SmartyStreets API
SmartyStreets is an address validator for U.S. postal addresses. Feed SmartyStreets an address, like “11 Church Street, Burlington VT” and, if the address is matchable with the official U.S. postal service address, it will be returned, including the 9 digit zip code. The SmartyStreets API has some of the best API documentation. Here is the PowerShell code to validate a single address.
<#Powershell Code to query SmartyStreets API Provide address validation for a single U.S. address submitted to the API LK 11/12/2014 #> $Output="" $Uri="https://api.smartystreets.com/street-address?"+ "street=11+Church+Street&"+ "city=Burlington&"+ "state=VT&"+ "auth-id=myauthid"+ "auth-token=myauth-token" $Output=Invoke-RestMethod -Uri $Uri -ContentType application/json -Method Get $Output.delivery_line_1 $Output.last_line $Output.metadata
Athe auth-id and auth-token are values that you obtain from the Smartystreets site, which validate your account.
The result of the code is placed in the variable $Output.
Running this program provides the following output, the two validated address lines, and a slew of meta-data related to the address, including the county, gps coordinates, etc.
11 Church St
Burlington VT 05401-4417
record_type : S zip_type : Standard county_fips : 50007 county_name : Chittenden carrier_route : C009 congressional_district : AL rdi : Commercial elot_sequence : 0196 elot_sort : A latitude : 44.47953 longitude : -73.21282 precision : Zip9 time_zone : Eastern utc_offset : -5 dst : True
You can see the returned fields by piping $Output to Get-Member The delivery_line_1, and last_line contain the validated address with nine-digit zip code.
PS >$Output | Get-Member
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
analysis NoteProperty System.Management.Automation.PSCustomObject analysis=@{dpv_match_code=Y; dpv_footnotes=AA...
candidate_index NoteProperty System.Int32 candidate_index=0
components NoteProperty System.Management.Automation.PSCustomObject components=@{primary_number=11; street_name=C...
delivery_line_1 NoteProperty System.String delivery_line_1=11 Church St
delivery_point_barcode NoteProperty System.String delivery_point_barcode=054014417112
input_index NoteProperty System.Int32 input_index=0
last_line NoteProperty System.String last_line=Burlington VT 05401-4417
metadata NoteProperty System.Management.Automation.PSCustomObject metadata=@{record_type=S; zip_type=Standard; ...
Pass parameters to executables from the PS command line.
Working from within the PowerShell ise sometimes you still need to run an external editor to edit a text file. I use Notepad++ for this purpose and it is a pain to invoke it from within the command line and to pass the name of the file that I want to edit at the same time. Here is how you can start NotePad++ by itself.
& "C:Program Files (x86)Notepad++notepad++.exe"
Note that the full command string is within double quotes, to accommodate the spaces in the folder names. Because the strings are quoted, we also need to to preface the string with the at “&” sign, which means “execute this….”
Now, typically I’ll have an existing text or html file that I already want to edit. This can be appended to the command.
& "C:Program Files (x86)Notepad++notepad++.exe" winners.txt
Notepad++ will open multiple files in separate tabs that are passed to it when opening.
& "C:Program Files (x86)Notepad++notepad++.exe" winners.txt,loosers.txt
More details are in this article at Windows IT Pro.
Powershell: Basic text processing
Problem: I’ve received two files of eMail addresses. These represent one group that were awarded a grant, and much larger group that were rejected. I’m going to call these two files winners.txt and losers.txt.
I want to end up with the following:
1. Eliminate invalid eMail addresses
2. Eliminate duplicate records in both files
3. Check to see that anyone in the winners file does not appear in the losers file.
Although I’ve received these as an Excel Spreadsheet, I’ll use Excel to write them out as comma-delimited text files. Maybe later I’ll figure out how to work with the Excel spreadsheet directly from Powershell.
After exporting the eMail addresses from Excel they are contained in two .csv files….comma delimited text with each field enclosed in quotes. There were a huge number of blank lines at the bottom of each file, which I’m assuming are a result of the Excel export.
Working with the winners.txt file:
1. Count the number of lines in the file:
Get-Content winners.txt | measure-object -Line
2. Eliminate the blank lines. These were lines that had an empty string enclosed in double-quotes followed by a carriage-return, line feed..
(Get-Content winners.txt -Raw)`
.Replace( "`"`"`r`n" ,"" )| Set-Content winners.txt
3. Eliminate the rest of the quotes
(Get-Content winners.txt -Raw).Replace( "`"" , "" ) `
| Set-Content winners.txt
4. Validate for the presence of an @ character in each line. (Ok, I know there are some more robust email format evaluation routines using regular expressions or even .NET objects, but I’m reserving those for another time. Right now, I’m assuming that if a line includes an at sign ‘@’ then I can work with it).
(Get-Content winners.txt) | select-string "@" `
| Set-Content winners.txt
5. Eliminate duplicate lines
$fn=winners.txt
( Get-Content $fn ) | sort | get-unique >$fn
Note that you have to sort the lines to be able to get-unique.
Having done the above four steps on the winners’ file, I repeated them with the losers’ file. Then I compared the two files to find lines that appeared in both files, (which of course meant that my winners’ eMail addresses were present in the loosers’ file.)
5. Find lines that appear in both files.
Compare-Object -DifferenceObject $winners `
-ReferenceObject $loosers `
-ExcludeDifferent
It turns out that all of my winners eMail addresses appeared in the loosers file as well. I deleted these with a manual search and replace… (but that’s another PowerShell story to look up).
Notes:
1. The parentheses around Get-Content in a command act like they do when doing arithmetic. They instruct PS to execute those commands first. This is necessary when modifying the conents of a file without creating an intermediary file to hold the contents. The downside is that the file is read entirely into memory, so there may be implications with especially large files.
2. The line continuation character is a back-tick character, “`”. It also serves as the escape character when you need to include a particular character as part of a search string. So, in the search string in step 1,Replace( “`”`”`r`n” ,”” ) means, replace any line that has “” as its contents.
3. Find the default printer
Get-WmiObject `
-Query " SELECT * FROM Win32_Printer WHERE Default= $true"
From <https://wordpress.com/post/78494701/4>
PowerShell: Shorten URLs with Google’s API
The following PowerShell code will return a shortened URL from a long URL using the Google link shortening URL API. Contrast this code with the code for Bit.ly. There are couple differences:
1. Calling the Google API is done with a POST. With Bit.ly it is a GET.
2. I’ve included an interactive prompt in the code below, that will get the long URL from the command line. Once the shortened link is printed, you can paste it to the clipboard. (Or…better yet, avoid mousing around, and pipe the result to the clipboard using the clip.exe utility
# Generate shortened URL using the Google API
# First time set up:
# * Log in with your Google login name and password
# * Go to the Google Developer Console at:
# * https://console.developers.google.com/project
# * Create a new project.
# * Obtain an application key.(Don’t worry about oAuth)
# * Be sure that you allow requests from
# * “all IP addresses”
$APIKey=”xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”
# Hard coded long link below,or comment out line 17 and
# uncomment 20-23 for an interactive command line
#######################################################
# Paste in the URL that you want to shorten here.
$LongURL=”http://twitter.com”
#
#######################################################
# $Instructions = “You can paste a long URL into the`
# command line by right-clicking the mouse.`n”
# $Instructions
# $LongURL= Read-Host “Enter or Paste in the Long URL”
#######################################################
#Convert the Long URL to a JSON key/value
$LongHash=@{“longUrl”=$LongURL}
$MyLongURI = $LongHash | ConvertTo-Json
# Make the API call
$MyShortURL=Invoke-RestMethod `
-Uri https://www.googleapis.com/urlshortener/v1/url?key=$APIKey `
-Body $MyLongURI `
-ContentType application/json `
-Method Post
# Print out the shortened URL
$LongURL
$MyShortURL.id
PowerShell FTP Follow-up
This script is an attempt to automate a lengthy error-prone copying and configuration process that we do each week. If we do the process manually it can take anywhere from five to twenty minutes, and it tends to have various points of failure. The piece below is actually just one portion of the process. The steps include:
1. Get the name of a new folder to be created on the server
2. Get the name of the file to be copied into the new folder
3. Using the two new names, build a text file which contains commands that will be fed into PSFTP
4. Call PSFTP and run the commands in the text file.
This PowerShell script uses Putty FTP to log into an FTP server, create a new folder, and copy a file to that folder from the local host. Note the the steps for making the folder and copying the file are contained in a Putty script called gwkprocess.scr. This secondary script is is used as input to the Putty program after Putty makes the connection. Those steps are typical FTP steps:
CD / topdirectory
MKDIR /new directory
CD /newdirectory
PUT myfile.png
<# Powershell Scripted FTP
LK 10.30.2014
Send a file to the eMail server via FTP.
Uses the Putty Secure FTP program PSFTP
#>
# $FTPFolder=’/home/web/html/store/images/fy2014/Kids-Shop’
# Note that the login credentials are in clear text!
# Enter the new folder name here.
$NewFolder = “20141101ks”
# The Picture file to be copied is located in
# C:UsersLarryPowershell
# and should be named, with the usual naming convention
$PicFile = “20141101ks-image.png”
#Note line wraps.
#Build the Putty Script file
“cd /home/web/html/store/images/fy2014/Kids-Shop”| Out-File -FilePath C:UsersLarryPowershellgwkprocess.scr -Encoding ascii
“mkdir $Newfolder” | Out-File -FilePath C:UsersLarryPowershellgwkprocess.scr -Encoding ascii -Append
“put $PicFile” | Out-File -FilePath C:UsersLarryPowershellgwkprocess.scr -Encoding ascii -Append
“ls” | Out-File -FilePath C:UsersLarryPowershellgwkprocess.scr -Encoding ascii -Append
# Call the putty program
.psftp myuser@192.168.214.103 -P 22 -pw mypassword -v -2 -b gwkprocess.scr
This starts PSFTP in the Powershell window, makes the connection and then executes the gwkprocess.scr steps. It then closes the connection. If there is a problem, PSFTP will print a failure message, but clearly there is room for more error checking on the front end.
The presumption is that the secondary script gets rebuilt with new file and folder names each time the script is run. Obviously, there are some refinements to be included, like interactive data entry of the file and folder names.
PowerShell: Shorten Links with the Bit.ly API
Isn’t it annoying….you have actually go to a web site to shorten your URL! Here’s how to do it in PowerShell, and you’ll get the shortened URL printed on the command line.
If you save this script in a folder, you can run it by navigating to the script, right-clicking and choosing “Run in PowerShell”.
# Generate shortened URL using the Bitly API
# To set this up:
# * You must create an account at Bit.ly, and obtain an $
# * authorization token.
# * Verify your Bit.ly account with an eMail that Bitly
# * sends to your account.
# * Obtain an authorization token at: https://bitly.com/a/oauth_apps
# Paste the authorization token here…
$OAuthToken=”xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”
$Instructions = “You can paste a long URL into the command line by` right-clicking the mouse.`n”
$Instructions
$LongURL= Read-Host “Enter or Paste in the Long URL”
# Make the call
$MyURL=Invoke-WebRequest `
-Uri https://api-ssl.bitly.com/v3/shorten `
-Body @{access_token=$OAuthToken;longURL=$LongURL} `
-Method Get
#Get the elements from the returned JSON
$MyURLjson = $MyURL.Content | convertfrom-json
# Print out the shortened URL
$MyURLjson.data.url
Pause