Category Archives: Tech_Friday

PowerShell: Shorten URLs with Google’s API

The following PowerShell code will return a shortened URL from a long URL using the Google link shortening URL API. Contrast this code with the code for Bit.ly. There are couple differences: 
1. Calling the Google API is done with a POST. With Bit.ly it is a GET. 
2. I’ve included an interactive prompt in the code below, that will get the long URL from the command line. Once the shortened link is printed, you can paste it to the clipboard. (Or…better yet, avoid mousing around, and pipe the result to the clipboard using the clip.exe utility 


PS >$MyShortURL.id | clip.exe 

3. If it doesn’t matter, and you are already using the Google API,  use Bitly; initial setup is a snap.  I worked out the Google version because we want to be able to track our shortened links using Google Analytics.  Links shortened by Google are automatically made into tracking links. 

4. The Google call uses the PowerShell commandlet Invoke-RestMethod. The Bitly call uses the Invoke-WebRequest commandlet.  

5. For troubleshooting, I used the HttpRequestor FoxFire plugin o make sure I was making the correct API call.  


# Generate shortened URL using the Google API 
# First time set up:  
# * Log in with your Google login name and password  
# * Go to the Google Developer Console at:
# * https://console.developers.google.com/project
# * Create a new project. 
# * Obtain an application key.(Don’t worry about oAuth)
# * Be sure that you allow requests from 
# * “all IP addresses” 

$APIKey=”xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”

# Hard coded long link below,or comment out line 17 and 
# uncomment 20-23 for an interactive command line 

#######################################################
# Paste in the URL that you want to shorten here. 
 $LongURL=”http://twitter.com” 
#
#######################################################
# $Instructions = “You can paste a long URL into the`
# command line by right-clicking the mouse.`n”
# $Instructions 
# $LongURL= Read-Host “Enter or Paste in the Long URL”
#######################################################

#Convert the Long URL to a JSON key/value 
$LongHash=@{“longUrl”=$LongURL}
$MyLongURI = $LongHash | ConvertTo-Json

# Make the API call
$MyShortURL=Invoke-RestMethod `
-Uri https://www.googleapis.com/urlshortener/v1/url?key=$APIKey `
-Body $MyLongURI `
-ContentType application/json `
-Method Post

# Print out the shortened URL 
$LongURL
$MyShortURL.id

BrightPearl API Part V – PowerShell

PowerShell, is a Microsoft download, currently at version 4.0.  It is a batch command language and enhanced command shell which is the current successor to the CMD.exe found in Windows. It seems to be a bit of a mash-up between the old DOS command line, the Unix-style shells like BASH, and includes connections to .NET objects. Unfortunately, it also abstracts a number of parameters to objects, such that you can’t just put stuff on a command line; you have to assign it to an object parameter when passing parameters, or when getting things out from a returned object. 
 
To continue with our Brightpearl example:
Recall that when working with the Brighpearl API, you have to first obtain a temporary authorization code, which is good for roughly 30 minutes. You do this by making an HTTP POST which includes your credentials in the body of the post.
 
Obtain a Brightpearl Authorization Code:
 
This is a two step process:
 
1. First stuff the authorization credentials into a variable. Note that the credentials are formatted in nested JSON notation.
$bpauth = “{apiAccountCredtentials:{emailAddress:”myemail@mycompany.com”, password:”Mypassword”}}”
2. Execute the command line call using the stored log-in credentials in the Body parameter
http://ws-use.brightpearl.com/mybpaccountname/authorise -Body $bpauth -ContentType application/json -Method Post
The Brightpearl server returns an authorization code which is displayed as an object with a series of parameters. The authorization code is contained in the a JSON string in the “Content” parameter.  This is the authorization code that must accompany any subsequent call to the Brightpearl API.
StatusCode        : 200
StatusDescription : OK
Content           : {“response”:”53145c429-x1xx-y2sf-z34a-8abc9cde96f9gh”}
RawContent        : HTTP/1.1 200 OK
                    Pragma: no-cache
                    ruid: a745efb3-2414-428f-8427-5001e3c810b8
                    Connection: keep-alive
                    Content-Length: 51
                    Cache-Control: no-cache, must-revalidate
                    Content-Type: application/json;char…
Forms             : {}
Headers           : {[Pragma, no-cache], [ruid, a745efb3-2414-428f-8427-5001e3c810b8], [Connection, keep-alive], [Content-Length, 51]…}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 51
To isolate the code itself, you have to assign the output of the above command to a variable, and then access it using dot notation.
So, restate the above command to assign the output to a variable:
$bpcode = Invoke-WebRequest -Uri http://ws-use.brightpearl/microdesign/authorize -Body $bpauth -ContentType application/json – Method Post
If you then execute $bpcode.Content, then you’ll get the authorization code, (again in JSON format).
PS C:UsersLarryPowerShell>$bpcode.Content
This returns:
  {“response”:”53145c429-x1xx-y2sf-z34a-8abc9cde96f9gh”}
To see this in a more readable format:
PS C:UsersLarryPowerShell>$bpcode.Content | ConvertFrom-Json
response
——–
53145c429-x1xx-y2sf-z34a-8abc9cde96f9gh
To assign the authorization code to a variable itself, we either have to strip it from the JSON code, or from the returned converted version.  It probably is easier to convert via the JSON code because we can do a text search everything after the colon, and then strip off the double quotes and the last curly bracket.
$bpstring=$bpcode.content
The following command strips off everything up to the start of the actual code.
 
$bpstring = $bpstring. Trimstart( “{`”response`”`:” )
It gives us:   
“53145c429-x1xx-y2sf-z34a-8abc9cde96f9gh”}
The following command strips off the final quote and curly bracket.
$bpstring = $bpstring. Trimend( “`”`}”)
This gives us our final result; what we’re really looking for:
53145c429-x1xx-y2sf-z34a-8abc9cde96f9gh

Final Script:

The final script is pretty close to the interactive commands entered at the PowerShell command line. The main exception is the addition of the back tick escape character within the script. If you use the PowerShell Interactive Shell as an editor,  it will color code things nicely and flag scripting errors. 
# BrightPearl API: Get an authorization code for subsequent API queries
# Note escape character is the “`” (back tick), instead of the usual backslash.
# Double quotes need to be escaped when nested inside.
# LK 10/9/2014
 
 
# Assign credentials to an authorization object.
$bpauth = “{apiAccountCredentials:{emailAddress:`”myeMailAddress@mycompany.com`”,password:`”myPassword`”}}”
 
# Execute the HTTP POST to retrieve the authorization code. The result is assigned to the string $bpstring
# Note use of the back tick as a line continuation character
$bpstring = Invoke-WebRequest `
-Body $bpauth `
-ContentType application/json `
-Method Post
 
$bpAuthCode =$bpstring . Content
$bpAuthCode =$bpAuthCode . Trimstart( “{`”response`”`:” )
$bpAuthCode =$bpAuthCode . TrimEnd( “`”`}” #Note escape codes for the search expression
 
# Print the Authorization Code
$bpAuthCode 

Brightpearl API Part III: Working with Resources

This is the third posting regarding the Shopify / Brightpearl API
Part I is located here.
Part II is located here.

The Brightpearl API describes resources that can be queried using an HTTP GET request. Resources look suspiciously like tables in a database.  The Brightpearl resources include: 

Orders
Contact
Return contact information for a single contract with the ID of #200 
GET https://ws-use.brightpearl.com/2.0.0/myaccountid/contact-service/contact/4 
Postal-Address
GET https://ws-use.brightpearl.com/2.0.0/myaccountid/contact-service/postal-address/107
Recall that each of the GET calls above require a valid authorization code to be included in the call as a header 
brightpearl-auth: xxxxxx-xxxxxxx-xxxxxxxx-xxxxxxx-xxxxxxxxxx
Also, that you’d replace myaccountid with the name of your Brightpearl account. 
Since the order returns only the contact ID,  we need to go to contacts to get the person’s name. 
Since the contact only returns the person’s first and last name, we need to go to the address object to get the address. 
So one way that this might work….    
Dimension an array arWorldShip[10], which will hold the following pieces of information: 
order number, contact id, firstName, lastName, address id, address1, address2, address3, address4, postal code, 
Query to get the orders for the day, and then loop through each order 
For each shop order:  
     Store the order number in an arWolrdShip[1]  and the contact ID in arWorldShip[2]
     Query for the customer – to get the first and last name, and address ID
     Store these in the array 
     Query for the address – to get the address information.  (oh and check…for shipping/vs. postal address). 
     Store these in the array 
Next shop order 
Write out the array variables as a record; converting to .DBF.  

Web Services, REST, Shopify and Brightpearl Part I

Part I.

Background:

I am currently working on a project which involves a Shopify online web store, and the Brightpearl Inventory and CRM system. Both of these cloud-based systems have an Application Programmer’s Interface, (API) which provide a programmatic way to query and manipulate the data that has been entered via the normal web interface. They use these APIs to talk to each other and make them available to programmers who want to create custom functionality or plugins for the systems. Communication with these APIs can be done using a REST compatible client written in PHP, Python, Ruby on Rails, or a host of 3rd-generation languages like C# and Visual Basic.
REST stands for Representational State Transfer. This is the most recent flavor of network programming, similar to SOAP, XML, and XML-RPC, and even good old remote procedure calls.

Use-Case:

I’m looking into a way to extract data from the Brightpearl inventory system; I want to query for each day’s purchases and extract the order number, customer name and shipping information. I want to take this information and format it as an .DBF file for use by the UPS WorldShip program. Note that in this example, I’m interested in being a client of an existing web service, and, for the moment I really just need to query the service for existing data, I don’t need to add or delete records on the server.
To start this odyssey, I’m using my Windows workstation. I’m thinking eventually if I need to have a web server for testing (to run PHP or RAILS for example), that I’ll spin that up as a virtual machine using VirtualBox on Windows with Ubuntu Server as my guest OS with a mySQL backend.
The Brighpearl documentation suggests several tools that can be used to send requests to the API. Perverse as it sounds. I found it was helpful to install no less than three add-ons for FoxFire and Chrome to send the API requests, which enabled learning the mechanics of the process a little easier.
For Chrome:
For FireFox:
Each of these three add-ons allow you to send requests to a web server. Each is slightly different. The Chrome add-on includes a parser for JSON data, which is really helpful when you are working with JSON…which is the case with Brightpearl.
Brightpearl also suggests a book from O’Reilly called RESTful Web Services by Leonard Richardson and Sam Ruby. The book was published in 2007, so although it has some useful information, it is somewhat dated. There is nothing about oAuth in it for example.
 
To get started with the Brightpearl API, you have to make sure that your user account is authorized to work with the API. This is done by accessing the “Staff” under Setup, and making sure that there is a green checkmark next to the user’s name in the API access column. 
Get an Authorization Token
Brightpearl requires that you obtain an authorization token prior to accessing any other requests.  The request for the authorization token takes the form of  a POST request  which includes your user name and password in the request payload. The URI of the payload includes two variables,  your brightpearl server location, the name of your BrightPearl account and a Content-Type of text/xml
Content-Type: text/xml
where
use=”US East”
“microdesign”, is the name of your Brightpearl account id
The user name and password are passed as JSON name pairs to the apiAccountCredentials variable:
{

    apiAccountCredentials:{
        emailAddress:”myname@mydomain.com”,
        password:”mypassword”
    }

}
Note that the double quotes enclosing the eMail address and password are also present.
So, if you look at the raw request that is sent, the full request looks like this:
POST https://ws-use.brightpearl.com/microdesign/authorise
Content-Type: text/xml
{
apiAccountCredentials:{
emailAddress:”myname@mydomain.com”,
        password:”mypassword”    }
}
If the request is successful, you’ll receive a hexedicimal number back which is your authorization token.
{“response”:” xxxxxx-xxxxx-xxxxx-xxxxx-xxxx-xxxxxxxxxx”}
Once you have the authorization token, it is used in subsequent requests as a substitute for your user name and password. The token expires after about 30 minutes of inactivity…so you’ll have to issue another authorization request and obtain a new token after that time. 
Once you have gotten the authorization token, you can start making requests. The basic request is a “resource search” which is a query of the Brightpearl data. Resource searches are issued with GET requests, and must include the API version number. The authorization code is sent as a header along with the request. 
 
As a reminder, the authorization request is a POST, and the resource query is a GET.
(More on resource searches in Brightpearl).
GET https://ws-use.brightpearl.com/2.0.0/microdesign/warehouse-service/goods-note/goods-out-search
brightpearl-auth: xxxxxx-xxxxx-xxxxx-xxxxx-xxxx-xxxxxxxxxx
This request returns a list of the current goods-out notes (Brightpearl’s nomenclature for a packing slip or pick-list).
Example with results: 
The folllowing GET request shows the current orders.
brightpearl-auth: xxxxxx-xxxxx-xxxxx-xxxxx-xxxx-xxxxxxxxxx
This returns a list of current orders, in JSON format. The format shows the structure of the data first, and then the actual records.  Note that there are only three orders!
{“response”:{“metaData”:{“resultsAvailable”:3,”resultsReturned”:3,”firstResult”:1,”lastResult”:3,”columns”:[{“name”:”orderId”,”sortable”:true,”filterable”:true,”reportDataType”:”IDSET”,”required”:false},{“name”:”orderTypeId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderTypeNames”],”required”:false},{“name”:”contactId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”required”:false},{“name”:”orderStatusId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderStatusNames”],”required”:false},{“name”:”orderStockStatusId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderStockStatusNames”],”required”:false},{“name”:”createdOn”,”sortable”:true,”filterable”:true,”reportDataType”:”PERIOD”,”required”:false},{“name”:”createdById”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”required”:false},{“name”:”customerRef”,”sortable”:true,”filterable”:true,”reportDataType”:”STRING”,”required”:false},{“name”:”orderPaymentStatusId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderPaymentStatusNames”],”required”:false}],”sorting”:[{“filterable”:{“name”:”orderId”,”sortable”:true,”filterable”:true,”reportDataType”:”IDSET”,”required”:false},”direction”:”ASC”}]},”results”:[[1,1,207,4,3,”2014-09-18T14:15:50.000-04:00″,4,”#1014″,2],[2,1,207,1,3,”2014-09-29T13:20:52.000-04:00″,4,”#1015″,2],[3,1,207,1,3,”2014-09-29T13:25:39.000-04:00″,4,”#1016″,2]]},”reference”:{“orderTypeNames”:{“1″:”SALES_ORDER”},”orderPaymentStatusNames”:{“2″:”PARTIALLY_PAID”},”orderStatusNames”:{“1″:”Draft / Quote”,”4″:”Invoiced”},”orderStockStatusNames”:{“3″:”All fulfilled”}}}
If you use the “Advanced REST Client Application For Chrome, it will decode the above so that it is readable:
{
response:

{
metaData:

{
resultsAvailable3
resultsReturned3
firstResult1
lastResult3
columns:

[

9]

0:  

{
name: “orderId
sortabletrue
filterabletrue
reportDataType: “IDSET
requiredfalse
}
1:  

{
name: “orderTypeId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderTypeNames
requiredfalse
}
2:  

{
name: “contactId
sortabletrue
filterabletrue
reportDataType: “INTEGER
requiredfalse
}
3:  

{
name: “orderStatusId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderStatusNames
requiredfalse
}
4:  

{
name: “orderStockStatusId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderStockStatusNames
requiredfalse
}
5:  

{
name: “createdOn
sortabletrue
filterabletrue
reportDataType: “PERIOD
requiredfalse
}
6:  

{
name: “createdById
sortabletrue
filterabletrue
reportDataType: “INTEGER
requiredfalse
}
7:  

{
name: “customerRef
sortabletrue
filterabletrue
reportDataType: “STRING
requiredfalse
}
8:  

{
name: “orderPaymentStatusId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderPaymentStatusNames
requiredfalse
}
sorting:

[

1]

0:  

{
filterable:

{
name: “orderId
sortabletrue
filterabletrue
reportDataType: “IDSET
requiredfalse
}
direction: “ASC
}
}
results:

[

3]

0:  

[

9]

0:  1
1:  1
2:  207
3:  4
4:  3
5:  2014-09-18T14:15:50.000-04:00
6:  4
7:  #1014
8:  2
1:  

[

9]

0:  2
1:  1
2:  207
3:  1
4:  3
5:  2014-09-29T13:20:52.000-04:00
6:  4
7:  #1015
8:  2
2:  

[

9]

0:  3
1:  1
2:  207
3:  1
4:  3
5:  2014-09-29T13:25:39.000-04:00
6:  4
7:  #1016
8:  2
}
reference:

{
orderTypeNames:

{
1: “SALES_ORDER
}
orderPaymentStatusNames:

{
2: “PARTIALLY_PAID
}
orderStatusNames:

{
1: “Draft / Quote
4: “Invoiced
}
orderStockStatusNames:

{
3: “All fulfilled
}
}
}

A Database for Grant Research

I put together a grants database screen (click to view full size) to consolidate information for funding sources, and to track dates and interactions.

It is definitely an evolving project, but contains the basic information need to contact the funder, the deadline dates involved, the funder’s areas of interest, and the typical range of a grant award.

So far, I’ve been concentrating on foundation funding. Many foundations typically ask for a letter of interest before you put together a full proposal. So, I’ve included multiple date fields, a deadline for a letter of interest, a deadline for a full proposal, and a date when they announce their award.

Originally I thought that this database would be mostly for research, but after working with the online grants database, Grantstation, I think I will reserve this database for funders that I really expect to submit to. Some ideas for future enhancements include:

  • Links to standard “boilerplate” paragraphs that are used in an application. 
  • Links to edit the proposal or letter directly in Word. 
  • Links to the PDFs of the proposal. 
  • Reports that create a grants calendar. 
Before anyone comments that “you should really use X software” for this purpose, I just want to say that I’ve used several in the past, including DonorPerfect and Blackbaud, and evaluated many others. Right now, I’m in the process of rethinking my entire workflow automation from the ground up, and this very lightweight approach is just what I’m looking for. Plus its in FileMaker, so I can run it on my Windows machines at work, or my Macs at home.     

Rebootolator: Execute a Remote Linux Shell Script from Windows

Ok,  so, your mission, should you decide to accept it, is to restart mySQL and Apache on a remote server. This restarts a balky web site hosted by Apache, and also restarts a mySQL server which is used for a back-end for Drupal.

You want to execute this from your Windows computer.

The target computer runs CentOS 5.6 This is an (ancient) Red Hat Linux derivative, running (ancient) mySQL and Apache.

I ended up using PLink called from a Windows .CMD file to execute a bash shell script.  The shell script looks like this:

Rebootolator

#!/bin/bash -p
# Rebootolator – Reboots Apache and mySQL on a target Server
# LK Microdesign June 25, 2014
TERM=”xterm”
export TERM
clear
echo
echo ‘Rebooting Apache and mySQL on myServer’
echo ‘———————————–‘
echo ‘Restarting mySQL’
/etc/init.d/httpd restart
echo ”
echo ‘Restarting the Apache web server.’
/etc/init.d/mysqld restart
echo ‘Reboot procedure completed’


Note this script is not stored on the target server, but simply put in the same folder as the windows cmd file on my windows box.  

Now for the Windows command file: 

Reboot.CMD

:: Batch file to restart services on myServer
:: Restarts mySQL and httpd 
:: Uses the Rebootolator shell script
:: LK/Microdesign August 12, 2014  
@echo off
cls
echo. 
echo.
plink -ssh username@192.168.xxx.xxx -m rebootolator.sh -pw mypass
echo.
echo.

pause >nul | echo Press any key to exit. 

So, lets deconstruct the Windows Reboot.CMD file.
The first four lines are comment lines. Turns out, you can use two colons to preface a comment in Windows, (who knew?) instead of REM.
Line 5 turns off output to the screen.
Line 6 clears the screen.
Line 7 and 8 put in blank lines.
All the work happens on line 9, using the PLINK command. PLINK is the command line version of PUTTY, a free open source terminal program for Windows workstations. Both PLINK and PUTTY are pretty wonderful and highly recommended if you need to access Linux machines from Windows.
-ssh means “use the secure socket layer protocol to log into this machine”
username@192.168.xxx.xxx is a administrator’s account on the target machine,  probably the root account.
-m rebootolator.sh is the name of the shell script (above) that needs to run on the target machine.
-pw mypass is the password for the account used to log into the machine.

Deconstructing the Rebootolator.sh script:
#!/bin/bash -p  just means this is a BASH script
The two commands that actually restart the mySQL server, and the Apache server are: 
/etc/init.d/httpd restart
/etc/init.d/mysqld restart

The rest, (the echo commands) write out what  is happening at the command line. The Term command is my attempt to avoid a harmless error message that occurs when the script starts to execute.

Since I didn’t realize I could host the Rebootolator.sh script in my Windows folder, I originally though I’d have to log into one Linux box, and then execute the script on the target box.  Turned out the whole thing was simpler using PLINK, which is the equivalent of SSH and SSHPASS programs used to access remote machines from the Linux command line.

ClearOS: A Linux-based Windows SBS Replacement

In my ongoing project of trying to clone a Redhat Linux server, I ran across a help file that was written for an operating system called ClearOS. I assumed that this was another Linux distribution, and ignored it at first but then, while waiting for another installation, I spent some time reading the web pages.

ClearOS is a combination of a core Linux distribution based on Red Hat and CentOS. It includes a complete set of applications to provision an entire office. Perhaps the main advantage is that it takes what are usually a number of several different disparate Linux-based programs, and it puts a slick web-based management front-end them. ClearOS is very modular; you can make things as sophisticated or simple as you want.

To get a closer look, I downloaded and installed the ClearOS Virtualbox demo. (The only glitch was a problem with the 64-bit demo; I re-downloaded the 32-bit version and that installed perfectly on Virtualbox on my iMac.)

Setup is accomplished by a wizard that walks you through a sequence of steps to install the software, connect to the internet, configure the firewall and configure additional services.

If you want to see how ClearOS looks without worrying about the installation, you can “manage” a virtual server with a Live-Demo.

The screenshot shows options for backing up local workstations.

ClearOS offers a number of different versions and support levels. You can download and run the community edition for free, a choice that I might consider to replace a Windows SBS 2011 server if there are no processes on the server that are dependent on Windows. You can install it and run it on your own dedicated hardware. (They don’t recommend running the whole thing in a single virtual machine).

Or you can run it on a ClearOS hybrid appliance. These require the ClearOS Professional version which is a subscription-based support plan. The supported version can also be run on your own hardware. It includes certified and tested versions of all of the applications so that they are guaranteed to work together.

Years ago there was the Cobalt Qube, a single box which provided eMail, file and print services in a single cute box. (You can still find them on eBay). It was a great way to get an “instant network”, and I was sorry to see it discontinued. The ClearOS options provide a similar instant network, and would be suitable anywhere a Windows Small Business Server might be considered.