Tag Archives: Ubuntu

Install Ubuntu 16.04 on VirtualBox

The following is a step-by-step run through for installing Ubuntu 16.04 server as a virtual machine running under a Windows 10 host.  Two prerequisites:

Prepare the VirtualBox VM:

Click on “New” to create a new virtual machine: You’ll get this initial screen to choose the operating system you wish to install and choose a name for the your VM.

Ubuntu Install1.jpg

The next screen asks what you want to RAM memory. The recommended memory size is 768MB,  but I’ve had decent luck by boosting this to 4 gigs, (on my 8 gig Windows 10 workstation.).

UbuntuInstallMemory.jpg

Accept the next suggestion to create a virtual hard disk. The default 8GB is fine, because VirtualBox will expand this if necessary.

UbuntuHD.jpg

Accept the default next screen, to choose the file type.

VirtualHardDisk.jpg

… and accept the default “dynamically allocated”

DiskFixedSize.jpg

 

Finally you can choose the file size:

FileSize.jpg

In this case,  I chose 16 gigabytes.

Once you have completed the screens above, you need to change two other parameters before starting the actual installation:

Under Settings,  change the networking connection to “bridged adapter”

NetworkSettings.jpg

Under storage, point the little CD image to your .iso file.

StorageSettins.jpg

 

Install Ubuntu 16.04

At this point you are ready to start the VM, and go through installing Ubuntu from the .iso file.  This is the standard Ubuntu installation from here on out… run from a  console command line interface. Terminal.jpg

After making your keyboard and language selections, there are several prompts for additional information:

Choose a HostName: UBSandbox

Choose an initial account: larryk

Choose a login name for this account: larryk

Choose a password: mypassword

Encrypt your  home directory?  No

Set your time zone. Setup will suggest your local timezone and then ask
Is this time zone correct?   Yes

Partitioning Method:  Choose “Use Entire Disk”,  don’t worry about LVN

Select Disk to partition…. choose the defaults.

Write changes to disk? Yes

At this point the files are copied to the disk, and the installation continues ore or less on its own for five minutes or so,  then you’ll see a question about the use of an http proxy. This relates to the configuration for the package manager which is used to update the operating system. You can probably ignore this unless you know you are in a corporate environment that uses an http proxy server.

Proxy.jpg

The next screen asks you about updating. I would answer this with the default “no automatic updates”. .

Updates.jpg

Finally there is a screen that allows you to select additional software packages to be installed. I would include the LAMP server, and the OpenSSH server.  LAMP will be the usual Linux+Apache web server + PHP + mySQL

install.jpg

But wait! There’s more!   You will be asked for a password for the  mySQL database. Ignore this at your peril…and choose the same password as you used for your user account at the beginning of your installation.

mysqlpassword.jpg

At this point the installation will run for a few minutes and then …

Will this ever end?   Accept the default ‘Yes” to install the GRUB boot loader.

grubboot.jpg

And then….  we’re done.

alldone.jpg

At this point, you should have a working web server that is running an IP address on your network.  To figure out that address. run ifconfig from the VM console.  In our case we’re at 192.168.219.213

ifconfig.jpg

Looks promising.  Now,  we can open a web browser from our Windows workstation (or any other machine on the network, and we should see the Apache web server home screen.

Screenshot_111816_125839_PM.jpg

We’re ready install WordPress.  Before doing that however, you might create a snapshot* of the current state of the VM.  This means we will always have a backup of the current machine that we can fall back to as we’re experimenting with installing things. If you haven’t already installed Ubuntu three or four times, you can always delete the whole VM and reinstall if you want to start from square 1.

 

*I know….this is for a future blog post.

Install WordPress Using VirtualBox Part 2

Because WordPress is such a popular program, there are tons of resources available. After searching for installing WordPress on Ubuntu Server … I found this page.  I followed all of the instructions, with the two exceptions:

  • I did not configure a static IP address for the sandbox server.
  • I prefaced all commands with sudo.

 Once this was accomplished,  I ended up with the default installation page at:

192.168.219.212/wordpress/

wordpress_install

The first page that comes up is a page which asks for information which will will be written to the wp-config file.   Note that all of the parameters are identical to the ones that were used when setting up the mySQL database in the initial step.

wpsetup

The next page asks for the WordPress site name, and a login password.

wpinstall2

About 30 seconds later, you should see a message that WordPress was installed, and that you can now log in with the name and password that you just created in the last screen.  Do that and the familiar WordPress dashboard will come up.  Hooray!

wpdashboard

Install WordPress Using VirtualBox Part 1

Now that we’re WordPress experts,  I’m looking to get an instance of WordPress up and running to be able to experiment.  I’ve been trying this using VirtualBox on Windows 10.  I wanted to create a virtual web server using the latest Ubuntu server 16.04.

  1. Download the .iso file from Ubuntu
  2. Create a new virtual machine, “Ubuntu Server 16.04” within VirtuaBox. I gave it 4 gigs of RAM (half of the physical RAM in my workstation), and excepted all defaults for disk size, etc. The only difference afterwards is to change the Network Adapter from NAT to Bridged.virtualbox1

    3. Install Ubuntu. 
    During the Ubuntu installation, choose the option to install a LAMP server. This includes mySQL, the Apache web server, and the PHP language.

linux_install

4. Once Ubuntu is installed, the server will reboot. You will probably see that there are updates available.  So, to get these run the following commands:

sudo apt-get update
sudo apt-get upgrade

5. Figure out your local IP address:

ifconfig

If this address isn’t on your own subnet, then you can change the network specifications for the VM in VirtualBox from NAT to Bridged Adapter, then reboot the VM.

If all is well so far, you should see the Apache default home page when you type in the ip address into your browser.

apachedefault

At this point, we have a working web server running on our virtual machine.  Now we can actually install WordPress.  My first instinct for this was to use the apt-get method to install the files.

sudo apt-get install wordpress

This appeared to work but didn’t yield a running installation. So after searching I used these instructions to get to get a running WordPress installation.

 

Troubleshoot Apache with mod_status

We are having a bit of a contretemps with one our of Linode blog hosts. This host is running Ubuntu Linux 12X  LTS and WordPress, and seems to be having fits of high CPU utilization. One way to see into the web hosting process is to examine the Apache mod_status page.

Mod_status is an Apache module which prints statistics about the Apache application. By default the page will be accessible using the the suffix of /server-status on your root web site URL. http://mysite.com/server-status. However, by default this is restricted to a request from a browser at the 127.0.0.1 address.

If you log into the console, you can attempt to see the status by running the following:

apachectl status

or if you aren’t logged in as root,

sudo apachectl status

This returns the status page.

Screenshot_071516_011101_PM

If you are attempting to access this from a browser on another workstation via http://mysite.com/server-status, you get a 403 message:

Screenshot_071516_110946_AM

The first thing is to find where this restriction is located. It is going to be in one of the Apache configuration files. These are located within /etc/apache2. If the status module is enabled, there will be a status.conf file that can be edited.

sudo nano  /etc/apache2/mods-enabled/status.conf

Screenshot_071516_112041_AM

Edit the lines after the <Location /server-status> line, per the instructions. After editing mine looked like:

<Location /server-status>
   SetHandler server-status
   #Require local 
   Require ip 192.168.xxx.0/24
</Location>

where “xxx” is your local subnet.  In the above example, I’m accessing the server from inside the firewall from a workstation located on the same subnet as the Apache server. (In reality, I’m actually accessing a server running within a VirtualBox virtual machine, located on my Windows machine. )

Once these changes are made you have to restart Apache.

service apache2 restart

 

If this is successful,  you get a page similar to the character-based page, but nicely formatted in html. Since the data is similar, however, if there is any issue trying to get at the statistics, probably the character-based method at the console is the first thing to try.

Screenshot_071516_011525_PM

Ah…but what if you get a page, but it isn’t a status page?  This is the problem we have with our WordPress site, and it has to do with which page is served as the default. http://myblog.com/server-status  returns the default page from the blog rather than the server-status page.  Stay tuned for that fix.

WordPress: Troubleshooting Contact Forms

Down the rabbit holeI’m trying to get a contact form working on our WordPress site, and it is a little more complicated than I thought it might be.

The contact form plugin itself in WordPress. We’re using Contact Form 7. Documentation for Contact Form 7 says something to the effect “we’re sending upteen zillion eMails a day, and CF form “just works” for everybody.  Well, kinda, sorta. There are several other issues involved:

  • an MTA (mail transfer agent) which maybe (?)  has to be on the WordPress server
  • the php_mail() function which is part of PHP (the programming language for WordPress)
  • the wp_mail() function which may or may not call the php_mail() function
  • Restrictions on to and from addresses.

I wanted to just run everything through our Google Mail SMTP account, but that seems to add an additional layer.  So first I started with our blog site which is on another server, and tested whether the contact form worked there. It actually doesn’t,  that is, it appears to send mail correctly from the users’s perspective, but no email was received anywhere.

 

Per this note I checked to see if anything was listening on port 25, the usual SMTP port.

$ netstat -tanpl | grep :25
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 3030/master
tcp6 0 0 ::1:25 :::* LISTEN 3030/master

$ lsof -i :25
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
master 3030 root 12u IPv4 9667 0t0 TCP localhost:smtp (LISTEN)
master 3030 root 13u IPv6 9668 0t0 TCP ip6-localhost:smtp (LISTEN)

Looks promising.  it appears that both IP4 and IP6 are listening.

So, I created a basic PHP script to test at the server command line, and saved this as testphpmail.php. There are three parameters, the receiver’s address, the subject and the body of the message.

//This is a test of the basic PHP eMail function.

<?php
mail('joeblow@blow.com', "Test PHP Mail", "Test mail from PHP Mail");
?>

Run this script from the command line with the following command:

php -f testmail.php

This worked; I received the eMail. It shows that it comes from root.  At least  I knew that at least the fundamentals are in place on the server.

Checking the mail log,  at /var/log/mail.log I see the following:

$ cat/var/log/mail.log
May 20 14:28:17 li239-124 postfix/pickup[28709]: 11864F123: uid=0 from=<root>
May 20 14:28:17 li239-124 postfix/cleanup[28806]: 11864F123: message-id=<20160520142817.11864F123@localhost>
May 20 14:28:17 li239-124 postfix/qmgr[3039]: 11864F123: from=<root@localhost>, size=351, nrcpt=1 (queue active)
May 20 14:28:17 li239-124 postfix/smtp[28808]: 11864F123: to=<lkkg@myserver.org>,
relay=aspmx.l.google.com[173.194.204.27]:25, delay=0.21, delays=0.01/0.01/0.1/0.09, 
dsn=2.0.0, status=sent (250 2.0.0 OK 1463754497 e92si18328547qga.115 - gsmtp)

So this shows that the message was sent, using root@localhost as the sender’s address and my work eMail as the target.  And indeed, I received this email at work.

I checked the log file at /var/log/mail.log which presented various points of interest…including the fact that we are running the Postfix sendmail email server.  But there is also a status=bounced message for IP6.

$ cat /var/log/mail.log 
May 20 13:50:15 li239-124 postfix/pickup[13030]: 2973DF23B: uid=33 from=<www-data>
May 20 13:50:15 li239-124 postfix/cleanup[14078]: 2973DF23B: message-id=<49922f80371c574883b8b8d699f47d9b@blog.kidsgardening.org>
May 20 13:50:15 li239-124 postfix/qmgr[3039]: 2973DF23B: from=<www-data@localhost>, size=869, nrcpt=1 (queue active)
May 20 13:50:15 li239-124 postfix/smtp[14080]: 2973DF23B: to=<lkkg@myserver.org>, relay=aspmx.l.google.com[2607:f8b0:400d:c08::1a]:25, delay=0.41, delays=0.01/0.01/0.09/0.3, dsn=5.7.1, 
status=bounced (host aspmx.l.google.com[2607:f8b0:400d:c08::1a] said: 550-5.7.1 [2600:3c03::f03c:91ff:fe6e:916c] Our system has detected that this 550-5.7.1 message does not meet IPv6 sending guidelines regarding PTR records 550-5.7.1 and authentication. 
Please review 550-5.7.1 https://support.google.com/mail/?p=ipv6_authentication_error for more 550 5.7.1 information. a139si18154032qkc.140 - gsmtp (in reply to end of DATA command))
May 20 13:50:15 li239-124 postfix/cleanup[14078]: 8DAAAF23C: message-id=<20160520135015.8DAAAF23C@localhost>
May 20 13:50:15 li239-124 postfix/bounce[14081]: 2973DF23B: sender non-delivery notification: 8DAAAF23C
May 20 13:50:15 li239-124 postfix/qmgr[3039]: 8DAAAF23C: from=<>, size=3381, nrcpt=1 (queue active)
May 20 13:50:15 li239-124 postfix/qmgr[3039]: 2973DF23B: removed
May 20 13:50:15 li239-124 postfix/local[14082]: 8DAAAF23C: to=<www-data@localhost>, relay=local, delay=0.01, delays=0/0.01/0/0, dsn=2.0.0, status=sent (delivered to mailbox)
May 20 13:50:15 li239-124 postfix/qmgr[3039]: 8DAAAF23C: removed
root@li239-124:~# dpkg -S 'which sendmail'
dpkg-query: no path found matching pattern *which sendmail*.
root@li239-124:~# netstat -tanpl | grep:25
grep:25: command not found
root@li239-124:~# netstat -tanpl | grep :25
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 3030/master
tcp6 0 0 ::1:25 :::* LISTEN 3030/master

Just to be sure, I ran the PHP script again having it send to my home email address which is outside my work domain.  This bounced and I didn’t receive it at the home address.  Now,  in theory, this might not be a bad thing, since all of the Contact Form 7  messages from the blog page should be sent to an address within the same domain, like customerservice@myserver.com.

So far, then we’ve established:

  • PostFix is installed on the blog’s server.
  • Contact Form 7 shows that it is sending message, but the messages are bouncing.  (Hmm…I wonder what I’ve been missing all those times? )

This is so typical of IT. You think you’re troubleshooting an issue on one server, and upon investigating the same issue on another server that you think is working…you find that it isn’t and down the rabbit hole you go!

Web Services, REST, Shopify and Brightpearl Part I

Part I.

Background:

I am currently working on a project which involves a Shopify online web store, and the Brightpearl Inventory and CRM system. Both of these cloud-based systems have an Application Programmer’s Interface, (API) which provide a programmatic way to query and manipulate the data that has been entered via the normal web interface. They use these APIs to talk to each other and make them available to programmers who want to create custom functionality or plugins for the systems. Communication with these APIs can be done using a REST compatible client written in PHP, Python, Ruby on Rails, or a host of 3rd-generation languages like C# and Visual Basic.
REST stands for Representational State Transfer. This is the most recent flavor of network programming, similar to SOAP, XML, and XML-RPC, and even good old remote procedure calls.

Use-Case:

I’m looking into a way to extract data from the Brightpearl inventory system; I want to query for each day’s purchases and extract the order number, customer name and shipping information. I want to take this information and format it as an .DBF file for use by the UPS WorldShip program. Note that in this example, I’m interested in being a client of an existing web service, and, for the moment I really just need to query the service for existing data, I don’t need to add or delete records on the server.
To start this odyssey, I’m using my Windows workstation. I’m thinking eventually if I need to have a web server for testing (to run PHP or RAILS for example), that I’ll spin that up as a virtual machine using VirtualBox on Windows with Ubuntu Server as my guest OS with a mySQL backend.
The Brighpearl documentation suggests several tools that can be used to send requests to the API. Perverse as it sounds. I found it was helpful to install no less than three add-ons for FoxFire and Chrome to send the API requests, which enabled learning the mechanics of the process a little easier.
For Chrome:
For FireFox:
Each of these three add-ons allow you to send requests to a web server. Each is slightly different. The Chrome add-on includes a parser for JSON data, which is really helpful when you are working with JSON…which is the case with Brightpearl.
Brightpearl also suggests a book from O’Reilly called RESTful Web Services by Leonard Richardson and Sam Ruby. The book was published in 2007, so although it has some useful information, it is somewhat dated. There is nothing about oAuth in it for example.
 
To get started with the Brightpearl API, you have to make sure that your user account is authorized to work with the API. This is done by accessing the “Staff” under Setup, and making sure that there is a green checkmark next to the user’s name in the API access column. 
Get an Authorization Token
Brightpearl requires that you obtain an authorization token prior to accessing any other requests.  The request for the authorization token takes the form of  a POST request  which includes your user name and password in the request payload. The URI of the payload includes two variables,  your brightpearl server location, the name of your BrightPearl account and a Content-Type of text/xml
Content-Type: text/xml
where
use=”US East”
“microdesign”, is the name of your Brightpearl account id
The user name and password are passed as JSON name pairs to the apiAccountCredentials variable:
{

    apiAccountCredentials:{
        emailAddress:”myname@mydomain.com”,
        password:”mypassword”
    }

}
Note that the double quotes enclosing the eMail address and password are also present.
So, if you look at the raw request that is sent, the full request looks like this:
POST https://ws-use.brightpearl.com/microdesign/authorise
Content-Type: text/xml
{
apiAccountCredentials:{
emailAddress:”myname@mydomain.com”,
        password:”mypassword”    }
}
If the request is successful, you’ll receive a hexedicimal number back which is your authorization token.
{“response”:” xxxxxx-xxxxx-xxxxx-xxxxx-xxxx-xxxxxxxxxx”}
Once you have the authorization token, it is used in subsequent requests as a substitute for your user name and password. The token expires after about 30 minutes of inactivity…so you’ll have to issue another authorization request and obtain a new token after that time. 
Once you have gotten the authorization token, you can start making requests. The basic request is a “resource search” which is a query of the Brightpearl data. Resource searches are issued with GET requests, and must include the API version number. The authorization code is sent as a header along with the request. 
 
As a reminder, the authorization request is a POST, and the resource query is a GET.
(More on resource searches in Brightpearl).
GET https://ws-use.brightpearl.com/2.0.0/microdesign/warehouse-service/goods-note/goods-out-search
brightpearl-auth: xxxxxx-xxxxx-xxxxx-xxxxx-xxxx-xxxxxxxxxx
This request returns a list of the current goods-out notes (Brightpearl’s nomenclature for a packing slip or pick-list).
Example with results: 
The folllowing GET request shows the current orders.
brightpearl-auth: xxxxxx-xxxxx-xxxxx-xxxxx-xxxx-xxxxxxxxxx
This returns a list of current orders, in JSON format. The format shows the structure of the data first, and then the actual records.  Note that there are only three orders!
{“response”:{“metaData”:{“resultsAvailable”:3,”resultsReturned”:3,”firstResult”:1,”lastResult”:3,”columns”:[{“name”:”orderId”,”sortable”:true,”filterable”:true,”reportDataType”:”IDSET”,”required”:false},{“name”:”orderTypeId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderTypeNames”],”required”:false},{“name”:”contactId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”required”:false},{“name”:”orderStatusId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderStatusNames”],”required”:false},{“name”:”orderStockStatusId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderStockStatusNames”],”required”:false},{“name”:”createdOn”,”sortable”:true,”filterable”:true,”reportDataType”:”PERIOD”,”required”:false},{“name”:”createdById”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”required”:false},{“name”:”customerRef”,”sortable”:true,”filterable”:true,”reportDataType”:”STRING”,”required”:false},{“name”:”orderPaymentStatusId”,”sortable”:true,”filterable”:true,”reportDataType”:”INTEGER”,”referenceData”:[“orderPaymentStatusNames”],”required”:false}],”sorting”:[{“filterable”:{“name”:”orderId”,”sortable”:true,”filterable”:true,”reportDataType”:”IDSET”,”required”:false},”direction”:”ASC”}]},”results”:[[1,1,207,4,3,”2014-09-18T14:15:50.000-04:00″,4,”#1014″,2],[2,1,207,1,3,”2014-09-29T13:20:52.000-04:00″,4,”#1015″,2],[3,1,207,1,3,”2014-09-29T13:25:39.000-04:00″,4,”#1016″,2]]},”reference”:{“orderTypeNames”:{“1″:”SALES_ORDER”},”orderPaymentStatusNames”:{“2″:”PARTIALLY_PAID”},”orderStatusNames”:{“1″:”Draft / Quote”,”4″:”Invoiced”},”orderStockStatusNames”:{“3″:”All fulfilled”}}}
If you use the “Advanced REST Client Application For Chrome, it will decode the above so that it is readable:
{
response:

{
metaData:

{
resultsAvailable3
resultsReturned3
firstResult1
lastResult3
columns:

[

9]

0:  

{
name: “orderId
sortabletrue
filterabletrue
reportDataType: “IDSET
requiredfalse
}
1:  

{
name: “orderTypeId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderTypeNames
requiredfalse
}
2:  

{
name: “contactId
sortabletrue
filterabletrue
reportDataType: “INTEGER
requiredfalse
}
3:  

{
name: “orderStatusId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderStatusNames
requiredfalse
}
4:  

{
name: “orderStockStatusId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderStockStatusNames
requiredfalse
}
5:  

{
name: “createdOn
sortabletrue
filterabletrue
reportDataType: “PERIOD
requiredfalse
}
6:  

{
name: “createdById
sortabletrue
filterabletrue
reportDataType: “INTEGER
requiredfalse
}
7:  

{
name: “customerRef
sortabletrue
filterabletrue
reportDataType: “STRING
requiredfalse
}
8:  

{
name: “orderPaymentStatusId
sortabletrue
filterabletrue
reportDataType: “INTEGER
referenceData:

[

1]

0:  orderPaymentStatusNames
requiredfalse
}
sorting:

[

1]

0:  

{
filterable:

{
name: “orderId
sortabletrue
filterabletrue
reportDataType: “IDSET
requiredfalse
}
direction: “ASC
}
}
results:

[

3]

0:  

[

9]

0:  1
1:  1
2:  207
3:  4
4:  3
5:  2014-09-18T14:15:50.000-04:00
6:  4
7:  #1014
8:  2
1:  

[

9]

0:  2
1:  1
2:  207
3:  1
4:  3
5:  2014-09-29T13:20:52.000-04:00
6:  4
7:  #1015
8:  2
2:  

[

9]

0:  3
1:  1
2:  207
3:  1
4:  3
5:  2014-09-29T13:25:39.000-04:00
6:  4
7:  #1016
8:  2
}
reference:

{
orderTypeNames:

{
1: “SALES_ORDER
}
orderPaymentStatusNames:

{
2: “PARTIALLY_PAID
}
orderStatusNames:

{
1: “Draft / Quote
4: “Invoiced
}
orderStockStatusNames:

{
3: “All fulfilled
}
}
}