Tag Archives: JSON

JSON for PowerShell Part 1

JSON, otherwise known as “Javascript object notation” is a fundamental way of representing key/value pairs. In database terms this translates as a field name (the key) and the field contents (the value).

Let’s show a single contact record in JSON, with two phone numbers, work and home.

"Address1":"123 Anywhere Lane",
 "Address2":"Apartment 404",
 {"type":"home","number":"920 234-3424" },
 {"type":"work","number":"920 535-2312"  }

A couple of characteristics:

The complete JSON structure is enclosed in its own curly brackets
If there is more than one record, then the whole structure is contained with square brackets, and the individual records are separated by a comma. This applies to nested records too, as in the case of the two phone numbers for the individual.

Below is the notation for two contact records. In this case, the whole structure is enclosed in square brackets. The individual contact records are enclosed in curly brackets and the two records are separated by a comma.

 "Address1":"123 Anywhere Lane",
 "Address2":"Apartment 404",
"PhoneNumbers":[ {"type":
"home","number":"920 234 3424"},
"type":"work","number":"920 535-2312"}]
"Address1":"123 Anywhere Lane",
"Address2":"Apartment 404",
"PhoneNumbers":[ {"typee":
"home","number":"920 234-3424"},
{"type":"work","number":"920 535-2312"}]

When writing JSON, its helpful to have an automatic checker for syntax. One such is JSONLint at http://jsonlint.com. In Visual Studio Code you can use the Shift-Alt-F within the editor to nicely format JSON code.

The rules of JSON are summarized at JSONLint, but basically include:

  • Keys are enclosed in double quotes.
  • All data elements that are not a boolean or integers are enclosed in double-quotes.
  • Individual entities (i.e. records) are separated by a comma
  • Each key/value pair is separated by a comma
  • Each entity is enclosed by curly brackets.

Since I’m editing JSON in Visual Studio Code, there is more how VSC handles JSON in the Microsoft docs at JSON editing in Visual Studio Code.

Powershell has a two commands that deal directly with JSON.

PS M:\PSFolder> get-command *json*     

CommandType     Name                    Version    Source
-----------     ----                    -------    ------
Cmdlet          ConvertFrom-Json    Microsoft.PowerShell.Utility
Cmdlet          ConvertTo-Json    Microsoft.PowerShell.Utility

These convert Powershell objects to and from JSON notation. More on that in Part 2.

Brightpearl API: Add UPS Tracking Numbers

We have now been using our web store for about a month, and for the most part things have been going pretty smoothly. One issue has been sending orders to our warehouse, and we’ve got a pretty good Powershell script that creates a comma delimited text file (.csv) of order numbers and address information from queries to the Brightpearl API. This file is sent daily to the warehouse via FTP, and warehouse staff  import the orders into their UPS Worldship program.


The second half of this saga is to obtain the UPS Tracking number for each shipment. Once the shipment has been processed in UPS Worldship, a tracking number is generated and stored in the UPS Worldship record for that shipment. Worldship has an export function which will add the tracking number to a .csv file of order numbers and tracking numbers that we can use to update the order record in Brightpearl. The structure of this file, (which is completely customizable) is:

Order Number – In our case it is the Brightpearl sales order number
Tracking Number – from UPS. These look like “1Z 041 388 03 8331 4101”
Expected Delivery Date.
The .csv file looks like this. (UPS loves long field names).


The next step is to walk through the .csv file, find an order number, and update the custom field PCF_TRACKING in Brightpearl to contain the tracking number. Here is the Powershell call to update a single record:

PS>$BPOrders=Invoke-RestMethod `
 -Uri http://ws-use.brightpearl.com/public-api/myBPAccount/order-service/order/100541/custom-field `
 -Body $body `
 -Headers $headers `
 -Method Patch

There are a couple points of interest here. For the most part it is “standard” Powershell syntax for the Invoke-RestMethod.
1. We invoke this by assigning the results of the API call to $BPOrders
2. The call has several lines; the line continuation character is a accent aigu or back-tick.
3. Note that this query uses a $headers variable which includes the two validation properties for the Brightpearl query: , the name of the application and the security token for the application. These are stored as a hashtable.

brightpearl-app-ref myappreference
brightpearl-staff-token mystaff-tokenXYZ123

More on obtaining the authcode here.

4. The $body variable is also created as a hashtable, but then converted to JSON, and placed between square brackets. This variable contains three parameters, the operation that you are performing on the record, the field that you want to modify, and the value that you want to put in the field. The syntax below simply says, “Replace the contents of the /PCF_TRACKING field with the value of 12345”.

$body=($body | ConvertTo-JSON)

The result is:

"op": "replace",
"path": "/PCF_TRACKING",
"value": "12345"

5. The -Method parameter is a “Patch”. This allows you to replace the contents of a single field in a record rather than replace an entire record as happens when you use PUT.

6. Finally note in the Invoke-RestMethod call, the URI contains “custom-field”. This is a literal, it isn’t the name of your custom field. The name of the custom field is contained in the body.I In the example above, it is “/PCF_TRACKING”

The above API call will replace the contents of a single field in a single record. The next step is to be able to loop through the .csv file, and for each record, find the corresponding record within the Brightpearl database, and update its Tracking number field.

Oh, one more thing, the results of the operation are contained in $BPOrders. The API actually returns the contents of ALL custom fields. You can choose which ones you want to see using dot notation.


Powershell: Limit API iterations in a single call


Many APIs limit the number of iterations that you can make in a single API call. For example, Brightpearl limits you to getting information for a maximum of 200 orders in a single API call. If you place a call with more than 200 orders, it will simply return an error message. SmartyStreets also places a limit of 100 addresses that you can validate with a single API call.


Dave Wyatt at PowerShell.org provides the following solution.

Lets assume there is an array of 1000 addresses which are returned by convertfrom-csv. Here are the first couple of records from the original .csv file.

PS>cat lapsed.csv
Joe Dokes,Joe, Mary,,,601 W 57TH St Apt 361,New York,NY,10019
Mary Smith ,Mary,Howard,,347 Poor Farm Rd,Colchester,VT,05446
Lu-Anne Jorden,Lu-Anne,Jess,,9603 North Kiowa Rd.,Parker,CO,80138

Here is the command that we use to read in the list into the variable $bigLlist

$bigList =(cat lapsed.csv | convertfrom-csv | 
Select-Object Addressee, Organization, Street, City, State, Zip )

$bigList is a custom object with the following layout:

PS>$biglist | get-member
TypeName: Selected.System.Management.Automation.PSCustomObject
Name         MemberType    Definition 
----         ----------    ---------- 
Equals       Method        bool Equals(System.Object obj) 
GetHashCode  Method        int GetHashCode() 
GetType      Method        type GetType() 
ToString     Method        string ToString() 
Addressee    NoteProperty  System.String Addressee=Kamal Aboul-Hosn 
City         NoteProperty  System.String City=New York 
Organization NoteProperty  System.String Organization= 
State        NoteProperty  System.String State=NY 
Street       NoteProperty  System.String Street=601 W 57TH St Apt 361
ZIP          NoteProperty  System.String ZIP=10019

If you look at this in Powershell, it prints out the contents of each record.

Addressee    : Joe Dokes
Organization :
Street       : 109 Fern Ct.
City         : Delray Beach
State        : FL
ZIP          : 33444

Addressee    : Mary Smith
Organization :
Street       : 205 Dorado Dr
City         : Cherry Hill
State        : NJ
ZIP          : 08034

Addressee    : Lu-Anne Jorden
Organization :
Street       : PO Box 81666
City         : Fairbanks
State        : AK
ZIP          : 99708

Ok, so now we have the full list as an object. The list now needs to be subdivided into groups of 100.

$counter = @{ Value = 0 }
$groupSize = 100
$groups = $bigList | Group-Object -Property { [math]::Floor($counter.Value++ / $groupSize) }

The $counter variable is a hash table, initialized to zero.
The $groupsize variable is the size of the individual group that can be sent. In our example it is set to 100, for a maximum of 100 addresses to be sent at a time.
The $groups variable creates a custom object, with the following members:

PS>$groups | gm
   TypeName: Microsoft.PowerShell.Commands.GroupInfo

Name        MemberType Definition                                                      
----        ---------- ----------                                                      
Equals      Method     bool Equals(System.Object obj)                                  
GetHashCode Method     int GetHashCode()                                               
GetType     Method     type GetType()                                                  
ToString    Method     string ToString()                                               
Count       Property   int Count {get;}                                                
Group       Property   System.Collections.ObjectModel.Collection[psobject] Group {get;}
Name        Property   string Name {get;}                                              
Values      Property   System.Collections.ArrayList Values {get;}                      

If you print out the contents of $groups, you see the following list. (I’ve truncated for readability…)

Count Name     Group                                                                                                              
----- ----     -----                                                                                                              
  100 0        {@{Addressee=Kamal Aboul-Hosn; Organization=; ...
  100 1        {@{Addressee=Chandler Dawson; Organization=; ...   
  100 2        {@{Addressee=Sidsel Heney; Organization=; ...
  100 3        {@{Addressee=John Marchetti; Organization=; ...
  100 4        {@{Addressee=Jane Ramsey; Organization=; ...
   59 5        {@{Addressee=James Tulloh; Organization=; ... 

This shows that I have 559 names in the original file which has been divided up into 5 groups of 100 and one of 59 names.

The next and final step is to iterate through each group and make the API call.

foreach ($group in $groups)
    $littleList = $group.Group | ConvertTo-Json
$Output = Invoke-RestMethod -Uri $Uri -Body $littlelist -ContentType application/json -Method Post 

The steps are:
For each group
Convert the addresses in one group to JSON
Assign it to the variable $littlelist
Send the contents of $littlelist as the body of the API call.
End Loop.

PowerShell: Formatting Dates

In Brightpearl, if you want to select orders from a specific date, the date needs to be entered in the format of YYYY-MM-DD. By default PowerShell returns dates in a format based on your “culture” setting; in the U.S. this means the default is MM-DD-YYYY. (Who made this up by the way?) In fact…a standard call to the Get-Date returns a “long date / time” string.

Thursday, November 20, 2014 12:46:19 PM

To just get the date, you add a couple parameters:

PS>Get-Date -DisplayHint Date -Format d

This returns the system date in the default culture format.

To transform this to YY-MM-DD, there is a parameter which takes a Unix format string.

PS>Get-Date -UFormat %Y-%m-%d

TechNet has a reference for all of the possible combinations and strings for the UFormat parameter.

Here’s the format for an order search using a hard coded date.
PS>$BPOrders=Invoke-RestMethod `
-Uri http://ws-use.brightpearl.com/public-api/nationalgardening/order-service/order-search?placedOn=2014-11-20 `
-Headers $headers `
-Method Get

We can put the date in a variable, and use that in the API call:

$Today=Get-Date -UFormat %Y-%m-%d

$BPOrders=Invoke-RestMethod `
-Uri http://ws-use.brightpearl.com/public-api/nationalgardening/order-service/order-search?placedOn=$Today `
-Headers $headers `
-Method Get

PowerShell: Shorten URLs with Google’s API

The following PowerShell code will return a shortened URL from a long URL using the Google link shortening URL API. Contrast this code with the code for Bit.ly. There are couple differences: 
1. Calling the Google API is done with a POST. With Bit.ly it is a GET. 
2. I’ve included an interactive prompt in the code below, that will get the long URL from the command line. Once the shortened link is printed, you can paste it to the clipboard. (Or…better yet, avoid mousing around, and pipe the result to the clipboard using the clip.exe utility 

PS >$MyShortURL.id | clip.exe 

3. If it doesn’t matter, and you are already using the Google API,  use Bitly; initial setup is a snap.  I worked out the Google version because we want to be able to track our shortened links using Google Analytics.  Links shortened by Google are automatically made into tracking links. 

4. The Google call uses the PowerShell commandlet Invoke-RestMethod. The Bitly call uses the Invoke-WebRequest commandlet.  

5. For troubleshooting, I used the HttpRequestor FoxFire plugin o make sure I was making the correct API call.  

# Generate shortened URL using the Google API 
# First time set up:  
# * Log in with your Google login name and password  
# * Go to the Google Developer Console at:
# * https://console.developers.google.com/project
# * Create a new project. 
# * Obtain an application key.(Don’t worry about oAuth)
# * Be sure that you allow requests from 
# * “all IP addresses” 


# Hard coded long link below,or comment out line 17 and 
# uncomment 20-23 for an interactive command line 

# Paste in the URL that you want to shorten here. 
# $Instructions = “You can paste a long URL into the`
# command line by right-clicking the mouse.`n”
# $Instructions 
# $LongURL= Read-Host “Enter or Paste in the Long URL”

#Convert the Long URL to a JSON key/value 
$MyLongURI = $LongHash | ConvertTo-Json

# Make the API call
$MyShortURL=Invoke-RestMethod `
-Uri https://www.googleapis.com/urlshortener/v1/url?key=$APIKey `
-Body $MyLongURI `
-ContentType application/json `
-Method Post

# Print out the shortened URL 

PowerShell: Shorten Links with the Bit.ly API

Isn’t it annoying….you have actually go to a web site to shorten your URL!  Here’s how to do it in PowerShell, and you’ll get the shortened URL printed on the command line.

If you save this script in a folder, you can run it by navigating to the script, right-clicking and choosing “Run in PowerShell”.

# Generate shortened URL using the Bitly API 
# To set this up: 
# * You must create an account at Bit.ly, and obtain an $ 
# * authorization token. 
# * Verify your Bit.ly account with an eMail that Bitly 
# * sends to your account. 
# * Obtain an authorization token at: https://bitly.com/a/oauth_apps
# Paste the authorization token here…


$Instructions = “You can paste a long URL into the command line by` right-clicking the mouse.`n”
$LongURL= Read-Host “Enter or Paste in the Long URL”

# Make the call
$MyURL=Invoke-WebRequest `
-Uri https://api-ssl.bitly.com/v3/shorten `
-Body @{access_token=$OAuthToken;longURL=$LongURL} `
-Method Get

#Get the elements from the returned JSON 

$MyURLjson = $MyURL.Content | convertfrom-json 

# Print out the shortened URL 


BrightPearl API Part V – PowerShell

PowerShell, is a Microsoft download, currently at version 4.0.  It is a batch command language and enhanced command shell which is the current successor to the CMD.exe found in Windows. It seems to be a bit of a mash-up between the old DOS command line, the Unix-style shells like BASH, and includes connections to .NET objects. Unfortunately, it also abstracts a number of parameters to objects, such that you can’t just put stuff on a command line; you have to assign it to an object parameter when passing parameters, or when getting things out from a returned object. 
To continue with our Brightpearl example:
Recall that when working with the Brighpearl API, you have to first obtain a temporary authorization code, which is good for roughly 30 minutes. You do this by making an HTTP POST which includes your credentials in the body of the post.
Obtain a Brightpearl Authorization Code:
This is a two step process:
1. First stuff the authorization credentials into a variable. Note that the credentials are formatted in nested JSON notation.
$bpauth = “{apiAccountCredtentials:{emailAddress:”myemail@mycompany.com”, password:”Mypassword”}}”
2. Execute the command line call using the stored log-in credentials in the Body parameter
http://ws-use.brightpearl.com/mybpaccountname/authorise -Body $bpauth -ContentType application/json -Method Post
The Brightpearl server returns an authorization code which is displayed as an object with a series of parameters. The authorization code is contained in the a JSON string in the “Content” parameter.  This is the authorization code that must accompany any subsequent call to the Brightpearl API.
StatusCode        : 200
StatusDescription : OK
Content           : {“response”:”53145c429-x1xx-y2sf-z34a-8abc9cde96f9gh”}
RawContent        : HTTP/1.1 200 OK
                    Pragma: no-cache
                    ruid: a745efb3-2414-428f-8427-5001e3c810b8
                    Connection: keep-alive
                    Content-Length: 51
                    Cache-Control: no-cache, must-revalidate
                    Content-Type: application/json;char…
Forms             : {}
Headers           : {[Pragma, no-cache], [ruid, a745efb3-2414-428f-8427-5001e3c810b8], [Connection, keep-alive], [Content-Length, 51]…}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 51
To isolate the code itself, you have to assign the output of the above command to a variable, and then access it using dot notation.
So, restate the above command to assign the output to a variable:
$bpcode = Invoke-WebRequest -Uri http://ws-use.brightpearl/microdesign/authorize -Body $bpauth -ContentType application/json – Method Post
If you then execute $bpcode.Content, then you’ll get the authorization code, (again in JSON format).
PS C:UsersLarryPowerShell>$bpcode.Content
This returns:
To see this in a more readable format:
PS C:UsersLarryPowerShell>$bpcode.Content | ConvertFrom-Json
To assign the authorization code to a variable itself, we either have to strip it from the JSON code, or from the returned converted version.  It probably is easier to convert via the JSON code because we can do a text search everything after the colon, and then strip off the double quotes and the last curly bracket.
The following command strips off everything up to the start of the actual code.
$bpstring = $bpstring. Trimstart( “{`”response`”`:” )
It gives us:   
The following command strips off the final quote and curly bracket.
$bpstring = $bpstring. Trimend( “`”`}”)
This gives us our final result; what we’re really looking for:

Final Script:

The final script is pretty close to the interactive commands entered at the PowerShell command line. The main exception is the addition of the back tick escape character within the script. If you use the PowerShell Interactive Shell as an editor,  it will color code things nicely and flag scripting errors. 
# BrightPearl API: Get an authorization code for subsequent API queries
# Note escape character is the “`” (back tick), instead of the usual backslash.
# Double quotes need to be escaped when nested inside.
# LK 10/9/2014
# Assign credentials to an authorization object.
$bpauth = “{apiAccountCredentials:{emailAddress:`”myeMailAddress@mycompany.com`”,password:`”myPassword`”}}”
# Execute the HTTP POST to retrieve the authorization code. The result is assigned to the string $bpstring
# Note use of the back tick as a line continuation character
$bpstring = Invoke-WebRequest `
-Body $bpauth `
-ContentType application/json `
-Method Post
$bpAuthCode =$bpstring . Content
$bpAuthCode =$bpAuthCode . Trimstart( “{`”response`”`:” )
$bpAuthCode =$bpAuthCode . TrimEnd( “`”`}” #Note escape codes for the search expression
# Print the Authorization Code

Brightpearl API Part II What is JSON?

Part II of a series about working with web APIs. 
Part I is located here
JSON (JavaScript Object Notation) is described at http://www.json.org/. It is a simple data interchange language which depends on two fundamental data structures:
Object: An unordered set of name / value pairs
Objects are delineated by curly brackets. Each name /value pair is separated by a colon, and pairs are separated by a comma.
Array: An ordered collection of  of values
Values in an array can be almost any data type, including string, number, array, true, false, null, or object.
Arrays are delineated by brackets. Array elements are separated by a comma.
So,  when we’re passing in the authorization credentials for our Brightpearl API, we’re passing in a nested set of name/value pairs as a single object:
Brightpearl returns query results as JSON as well. The first part of a the result is the description of the data….the second part is the actual data. (See Part 1 of this series for the full example).
The JSON.org page has links to JSON libraries for a plethora of programming languages and databases. While not listed on the page there is also a JSON module for FileMaker.  For mySQL there is a discussion of a couple ways of formatting the output of a mySQL query into JSON using Ruby, PHP, or using PHPmyAdmin.