Can you make a Raspberry Pi Security Camera?

This weekend’s geeky challenge: Making a Raspberry Pi Security Camera.

TL;DR – Here’s a basic, practically-ready-to-deploy, open-source Raspberry Pi security camera.

Brief

Mostly because I’m cheap (but partly to reduce the scope and therefore increase chance of success) I added a few constraints:

  • Periodic still image capture is OK (streaming video will be tackled in a future iteration)
  • Securely store images in the cloud (so I can check them when I’m away from home)
  • Be configurable
  • Support the cheap official Raspberry Pi camera modules, particularly the “NoIR” variants which come without the Infrared filter for night-time captures when complemented with an IR light pack

Let’s get started.

Design

After a quick napkin sketch it becomes clear we need to build 4 key parts:

  • A .NET Core console application.
    • This is our entrypoint, and using .NET Core lets us compile to the “linux-arm” runtime target, i.e. Raspbian.
  • A timer
    • To periodically take the photos. I have some ideas for other conditions or events that could trigger the camera to take a snapshot (such as a PIR sensor detecting heat) so I’ve implemented the timer as a “TimerTrigger“, which implements an ITrigger interface. That way it’s nice and extendable.
  • Some code that speaks to the Raspberry Pi camera module
    • Again I’ve written a “CameraImageSource“, which implements the IImageSource interface.
    • I also wrote a TestImageSource which loads an image from the current working directory for testing.
  • A client for the cloud storage API
    • I’ve chosen Google Drive because it offers 15GB free storage, a decent SDK that supports .NET Standard (1.3) and has good documentation.
    • As with the other components, this is written as an implementation of the “IDataStore” interface, so it’s easy to expand to other cloud storage providers in the future.

And this is how they all communicate:

Implementation

The code is here:

https://github.com/richstokoe/pispy

Once you’ve cloned that repository, there are some things we need to do before we can run the code.

Cloud Storage Credentials

To use the Google Drive API, you need an OAuth2 client ID and client secret. Follow these instructions to get a client ID and client secret, then hit the Download button in the “Credentials” area of the API console and save the file as “client_secrets.json” in the src/PiSpy/ folder of the repository you just cloned.

This file will be copied to the output directory when the project is built, per MSBuild instructions in the PiSpy.csproj project file.

Configure the Raspberry Pi Security Camera

If you want to change the default options, open up the appSettings.json file in the project.

Timer interval

You can change the interval between camera shots by modifying the Triggers:TimerTrigger:Interval setting. This is in milliseconds (seconds / 1000 so 60 seconds = 60000).

The default is 180 seconds (3 minutes).

Camera output directory

You can change the output directory by modifying the CameraModule:OutputPath setting.

The default assumes you will copy the console app to /home/pi/pispy, if you want to copy it somewhere else, change the path accordingly, otherwise you’ll see an error from the “mmal” process.

Raspbian with a Desktop

For now, we need an operating system on the Pi with an interactive desktop to complete the Google OAuth2 authorization flow.

Follow the steps in my last blog to set up a Pi with Raspbian and the .NET core runtime – but with one minor difference: grab Raspbian Stretch, NOT Raspbian Stretch Lite. This gives you a Desktop – that’s important for when the Google authorization flow pops open a browser window to enter your Google account details.

(In a future iteration we’ll add a Kestrel HTTP endpoint to the service to negate the need for the non-Lite version of Raspbian with a desktop).

You can either plug in a HDMI-capable screen and keyboard, or you can enable VNC by running:

sudo raspi-config

at the command line (or via SSH) and enabling ‘VNC’ under the ‘Interfacing options’ menu.

I also found I had to set Chromium as my default browser for the authorization flow to work properly – to do that navigate to chrome://settings, choose “Set Chromium as my default browser” and then restart the pi by running:

sudo reboot

Deploy and run

As in my last blog, build the project targeting the linux-arm runtime:

dotnet publish -r linux-arm

and copy the the bin/Debug/netcoreapp2.0/linux-arm/publish folder via FTP to the pi. I copied it to /home/pi/pispy.

We’re going to need access to Raspbian’s PIXEL desktop in a moment, so connect to the Pi’s VNC server by following the instructions here. Once you’re in, open a Terminal and navigate to the folder you copied the /publish folder to. Create a new folder within it called stills, which is where the CameraImageSource code will write the pictures to before handing us a Stream.

cd /home/pi/pispy
mkdir stills

(I’m cheating a bit by simply running the “raspistill” executable that comes with Raspbian to take the pictures. More info here.)

Next, start the service by running:

dotnet /home/pi/pispy/PiSpy.dll

After the time specified in the Triggers:TimerTrigger:Interval appSetting elapses, a photo will be taken and then the GoogleDriveDataStore will trigger the authorization flow (this only needs to happen once). Once you’ve logged in future photos will stream up to the Google Drive.

Next steps

Pull requests are not only welcome, they are encouraged.

A good place to start: I haven’t had much time this holiday season to debug (same excuse for shoddy blogging 🙂 ), but there appears to be an async bug in the TimerTrigger whereby the subscribed actions are invoked on a separate thread and the Timer is restarted even though the CameraImageSource still hasn’t finished taking its picture.

And don’t forget to turn off your LEDs:

I hope you’ve enjoyed this slightly rushed guide to building a Raspberry Pi security camera!

Running an ASP.NET Core 2.0 app on Raspbian Stretch Linux on a Raspberry Pi with HTTPS

Today’s challenge: Serve a public API over HTTPS from a Raspberry Pi. I’ll follow up with an article about containerising the app and running it from docker on the Pi.

This article is a fast-paced guide to getting started without stopping to dwell on the details.

Some details will be mopped up in the last section for those wanting to know more.

Prepare the Pi

On your development machine (you won’t need to interact with the Pi directly, so no 2nd keyboard or monitor required):

  • Grab an 8Gb or larger microSD card and use Etcher to flash the Raspbian Lite image to the card.
  • Create an empty file in the root of the microSD card called “ssh”, with no extension, this will enable ssh on Raspbian.
  • Put the microSD card in the Pi, plug in a network cable and then plug in the power
  • After a minute attempt to ping the pi:
ping raspberrypi
  • If you get a response, ssh into the pi from a Bash shell (on Windows, you can use the bash shell that comes with Git):
ssh pi@raspberrypi
  • The “pi@” means you’re logging in as the “pi” user account. The default password is “raspberry”.

Installing the .NET Core prerequisites

This bit looks hard but is quite easy (thanks Dave).

At the ssh prompt, install the .NET dependencies by running each of the following commands (you can copy and paste these commands straight into bash):

sudo apt-get install curl libunwind8 gettext
curl -sSL -o dotnet.tar.gz https://dotnetcli.blob.core.windows.net/dotnet/Runtime/release/2.0.0/dotnet-runtime-latest-linux-arm.tar.gz
sudo mkdir -p /opt/dotnet && sudo tar zxf dotnet.tar.gz -C /opt/dotnet
sudo ln -s /opt/dotnet/dotnet /usr/local/bin

you can check if the .NET Core runtime has been installed by running:

dotnet --help

You should now see some the runtime command line options.

Configuring the App

If you don’t already have an ASP.NET Core app:

  • Download the SDK for your platform (Mac, Linux, Windows) from https://dot.net and install it
  • Make a new directory and navigate to that directory at the command line.
  • Run:
dotnet new react

By default, a new ASP.NET Core application will be set to only listen to requests only from the TCP/IP loopback address (127.0.0.1 or localhost), so:

Open up Program.cs in your ASP.NET Core app and change the BuildWebHost method to add the following:

public static IWebHost BuildWebHost(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .UseKestrel(options => {
            options.Listen(System.Net.IPAddress.Any,
                5000,
                listenOptions => {
                    listenOptions.UseHttps("raspberrypi.pfx", "<your-password>");
                }
            );
        })
        .UseStartup<Startup>()
        .Build();

This means that when the app runs on the Pi it will accept requests to port 5000 from external clients.

Please note, Kestrel is not a supported edge server, it is designed to run behind a reverse proxy such as nginx, Apache HTTP Server or Microsoft IIS when exposed to the outside world. Read and understand this before you open up your Pi to the big bad interwebs.

Deploying the App to the Pi

I’m not offering DevOps perfection here, I’m afraid we’re just going to FTP the app across to the pi. But first we need to compile the app so it works with the Raspberry Pi’s low-power ARM processor.

On your development machine, drop to the command line, navigate to your project directory and publish your app so it works on Raspbian by executing the following command:

dotnet publish -r linux-arm

This creates a bin/Debug/netcoreapp2.0/linux-arm/publish directory that contains the binaries for your ASP.NET Core app.

Grab your favourite FTP client. If you don’t have one, FileZilla will do the trick.

Connect your FTP client to your Pi by entering the following details:

Host: raspberrypi
Username: pi
Password: raspberry
Port: 22

The FTP client should show you the directory structure on the Pi. Copy the contents of your linux-arm/publish directory to any path on the pi (I chose /home/pi/piservice/) using the FTP client.

HTTPS

We’re going to use a self-signed certificate to show HTTPS is possible. In a real-world scenario you’d sign a relatively short-lived RSA keypair with a certificate signed by a trusted root cert (and also, you’d probably not use a Raspberry Pi and publicly-exposed Kestrel Web Server to run your services, but hey ho).

SSH into your Pi again and run the following command to create a public and private key pair that will be valid for a year.

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365

Next, generate a .PFX file from the two generated PEM files (thanks to Pete S Kelly):

openssl pkcs12 -export -out raspberrypi.pfx -inkey key.pem -in cert.pem

This creates the “raspberrypi.pfx” file that your ASP.NET Core app now refers to in Program.cs.

Running the app

At the SSH prompt, tell the .NET Core runtime on teh pito start your app:

dotnet <yourappname>.dll

<yourappname> will be the name of the project you created if you used Visual Studio, or the name of directory you created that you ran dotnet new in. Typically this will be something like HelloWorld.dll or Acme.Web.dll, etc.

If all has been successful you’ll see the following echoed in your SSH session on the Pi:

pi@raspberrypi:~/piservice $ dotnet src.dll
Hosting environment: Production
Content root path: /home/pi/piservice
Now listening on: https://0.0.0.0:5000
Application started. Press Ctrl+C to shut down.

(My emphasis). From your development machine you should now be able to begin making requests to your app hosted on the Pi, https://raspberrypi:5000/

You will get errors about the certificate being untrusted, this is expected as your development machine has no reason to trust the little $30 computer, but you can skip past them – or read the final section of Peter Kelly’s article to learn how to trust the Pi’s self-signed certificate.

Making it Public

This is where things turn a bit vague as it’s up to you how you set up your network.

At the most basic level you need to tell your router to send traffic to port 5000 on your Pi. This usually involves adding a Port-Forwarding Rule. You’ll need to know your Pi’s IP address to set up the rule, so it makes sense to either give the Pi a static DHCP lease. Please refer to your router’s user guide for specific information.

To call your Pi from the outside world you’ll need your router’s public IP address – of course it’s best if this is static, ask your ISP if this is an option – and then you can set up a domain name to point to this IP address.

You can get a proper SSL certificate for your domain name from LetsEncrypt, or any other certificate provider.

Note: You’ll get SSL certificate errors when using a self-signed certificate, and browsers may stop allowing access to sites where the certificate doesn’t match the public domain name.

Follow Up

I’ll follow up soon with an article on containerising the ASP.NET Core app and running it on Docker on the Pi.

 

Google Watch: Time to DuckDuckGo

You can't spell
You can’t spell “Don’t be evil” without “evil”! Coincidence?

Google do a lot of good things. They host free webfonts to make the web a nicer place to be. Their cloudy PaaS service, Engine Yard, gets rave reviews. Their maps are better than anyone’s, their mobile OS is the most popular in the world, and their photo hosting offer is second to none. But they can be very evil sometimes too.

Remember when Google forced you to sign up to Google+ to comment on Youtube videos, or stole your email passwords while they took pictures of your house and then “forgot” to delete it after they got found out and all the Governments told them to, or made you type extra characters to include all your words in their search, or when their CEO said there was no place for privacy and anonymity on the Internet?

* big breath*

Well they are at it again.

And I’ve had enough.

The Devil’s In The Detail

For the last few days I’ve been seeing this ‘privacy reminder’ popup whenever I go to Google (including by searching in Chrome’s address bar). And it stops you dead in your tracks. You have to read through all the legalese before it lets you search for pictures of cats. Well I just don’t have time for that, I need instant cat gratification now!.

That sounds so wrong.

Anyway, I had a quick scan through the privacy reminder and immediately smelled a rat… It all seems really un-evil at first, you can choose to switch off some of Google’s invasive behaviour by following the handy-dandy links in the privacy reminder itself. Wowzers! What a nice thing to do. I opted to switch off all the weird adverts-following-you-around settings. They’re here, in case you’re wondering.

But then I noticed it says these settings are just for this browser. Your other devices and PCs will still track the living crap out of you. Continue Reading “Google Watch: Time to DuckDuckGo”

aspnetcore.dll failed to load. The data is the error

This morning, while trying to debug our big ol’ web project in Visual Studio 2015 I encountered a problem – it held me up for a while so I wanted to quickly blog about the solution in case it hits you too. When hitting F5 to start debugging, Chrome launched but then immediately Visual Studio detached from IIS Express and showed the following error:

A process with the ID of <id> is not running

True enough, IIS Express wasn’t running…

Open Wide and Say ‘Ahh!’, Mr Windows

I ran a Repair on IIS Express 10.0 in case it was an issue with that, or the self-signed SSL certificate it uses to host web projects over a secure connection…. but still had the same problem.

I then created a brand new ASP.NET MVC 5 project and hit F5… but that ran fine. Hmm, curious. That let me know IIS Express was fundamentally OK, and the issue lay with the big ol’ web project.

Microsoft are usually pretty good at logging when things go wrong so I fired up eventvwr, the Windows Event Viewer, and saw the following  error being thrown by IIS Express:

The Module DLL C:\Program Files (x86)\Microsoft Web Tools\AspNetCoreModule\aspnetcore.dll failed to load. The data is the error

Strange… We don’t have any ASP.NET Core projects in that solution so why aspnetcore.dll is being loaded was beyond me. Furthermore, that path didn’t exist on my disk. Continue Reading “aspnetcore.dll failed to load. The data is the error”

Things I learned at my last job

Today I closed a chapter in my life. After nearly 4 years tenure at a company I wanted to reflect on the things I learned over that time.

I have been very lucky to have a few excellent – world-class even – mentors here who have taught me things that will stay with me for the rest of my life, and I wanted to share the reflection process with you in the hope you gain something valuable too.

Individual Success Isn’t Success

For a long, long time I  adopted the ‘aircraft oxygen mask’ approach to my career: I’ll get to where I want to be first, then I’ll help others. This company has taught me that isn’t the right thing to do.

ubuntumeme

My thinking was always “I’ll be in a better position to help others” once I hit my objectives, but that simply doesn’t work in practice: without respectful, cooperative development across your team(s), you risk yourself hitting your goals at all, and if you haven’t helped others hit theirs too, nobody wins.

Dare I use the management-bullshit-bingo term ‘synergy’?

My current role here is a technical leadership role – that means I don’t have people reporting to me but I do have authority over technology direction and a remit to ensure conceptual integrity of the solution. I have led project teams before, I have even run small businesses before, but being a leader in a larger company was new to me when I began this chapter of my life, and I wanted to be good at it.

I’ve seen all the memes about the difference between a boss and a leader but for some reason I struggled to enact the differences. However, after some time spent being (in retrospect) a terrible boss, some sage advice from one of those mentors made everything ‘click’, and I was given the mental tools to develop the techniques required to become a good leader instead. (Note, a good mentor won’t give you the answer, but the means of finding it on your own!).

boss-vs-leader

“Take people with you.”

So what does that look like in practice? Last year I was offered the chance to travel to our American HQ to present some new work to 1,500 customers. ‘Prestigious’ isn’t even close – this is a huge event, so compelling that our customers pay us to listen to our plans and roadmap. The trip dripped with a significant amount of attached ‘kudos’ and the opportunity to rub shoulders with the highest of the high in the business. Not only that – the opportunity to ask probing questions to 1,500 customers about our technology direction is such a rare occurrence it was unmissable. The old me would have started packing immediately. Continue Reading “Things I learned at my last job”

The Sharing Economy

Your house, yesterday. Probably.
Your house, yesterday. Probably.

The ‘Sharing Economy’ is disrupting established industries and sending huge, powerful incumbents into a tizzy. Uber and AirBnB have shaken the taxi and hotel sectors, shifting power, control and profits from the RadioCabs and Hiltons of this world and into the hands of ordinary citizens armed with nothing more than a smartphone and a mobile data plan.

The question on everyone’s lips is: which industry will be disrupted by the Sharing Economy next?

A couple of years ago, I was in Portland, Oregon, for meetings with some colleagues. One lunchtime, our discussion diverged from work topics to an issue plaguing our home-lives, an issue common to both the US and UK: the reduction in bin-pickup frequency.

It’s a hot topic.

Dude, Where’s My Trash?

We tossed around some ideas to solve our overflowing bins issues, to solve the problems caused by local authorities switching from weekly to two-weekly pick-ups, and to solve that awkward situation we have all faced: that middle-of-the-night walk of shame, bin-bag over shoulder, roaming the streets like a crazed, ferral cat to find a neighbour’s bin with a bit of space left in it to deposit last night’s curry leftovers and beer bottles.

What does this have to do with the Sharing Economy?

A lightbulb lit: why not create a location-aware, social app to help out? Share My Trashcan was born, $5 per bag, with a $1 kick-back to us, it scales and is simple. But then one of our team mentally cycled through a Lean Startup build-measure-learn cycle and developed the concept, discovering that communities can come together to buy a shared dumpster, which would provide even more space (some of which could be shared with other communities!) and would also be picked up weekly.

Share My Trashcan was dead, long live Share My Dumpster!

Later in the day (after doing some actual work) we revisited the idea and the real, Lean, magic happened: we pivoted.  Continue Reading “The Sharing Economy”

Introducing QuickModules for ASP.NET MVC

INTERNET ALL THE THINGS!
Do what he says! He has a giant cotton bud in his hand!

Like you I often spend my weekends writing REST APIs for fun. And like you I am often frustrated by the amount of server-side code I need to write just to expose some data to JavaScript. Boilerplate, boilerplate, boilerplate. Ugh! So this weekend, in between bouts of erratic and unsatisfying jetlag-sleep, I decided to try and simplify the task of exposing an API to JavaScript. Introducing “QuickModules for ASP.NET MVC”, available now as a NuGet package!

QuickModules

QuickModules provides you with a basic, modular framework for hanging API endpoints from. Internally using the Managed Extensibility Framework (MEF) to load modules, QuickModules gives you the ability to build encapsulated CRUD services without needing to write, or understand, the usual plumbing.

Consumer Experience

As a software developer, and a human (vaguely), I like an easy life, so I’m all about the consumer experience when publishing code or frameworks (see my JavaScript business-object CRUD framework at http://richstokoe.github.io).

In order to ease any concerns you may have I will run through all the steps required (there are very few) to get QuickModules into your MVC application.

Step 0: Open Visual Studio (Duh!)

Step 1: Create a new MVC project

quickmodules_1

 

Step 2: Choose an Empty, Basic, Internet, Intranet or SPA template (actually, any will do, but these make the most sense)

quickmodules_2

 

Step 3: Notice how barren your project is without QuickModules. 🙁 (I’ve chosen a Basic template here)

quickmodules_3

 

Step 4: Install QuickModules using NuGet: Right-click on the References node in your Solution Explorer tree and click on “Manage NuGet Packages”:

quickmodules_4

 

Step 5: Search for “QuickModules” and choose “Install”. (Don’t worry if the details and version numbers look a little different on your PC, this is a screenshot of an early work-in-progress package).

quickmodules_5

 

That’s it! Hit F5 to run your application. If you’ve chosen an Empty or Basic template you’ll probably get a 404 Not Found error. Try navigating to the sample “Nearby Restaurants” module by adding /QM/Get?ActivityType=NearbyRestaurants to your URL (QM being shorthand for QuickModules of course!).

You should see a list of food places from The Simpsons as a wonderful, unformatted, JSON object array, ready to be consumed by your JavaScript:

quickmodules_7

 

But Wait There’s More!

Built into the framework (and optionally supported by your modules) is paging, and filtering. All for the low, low price of $9.95.  (Just kidding, it’s free). Use the LINQ-style Skip and Take arguments to control paging:

quickmodules_8

 

And use the Filter argument to narrow the result set down:

quickmodules_9 Continue Reading “Introducing QuickModules for ASP.NET MVC”

“The Format of the PE Module is Invalid” After VS2012 Update 2 Installed

If you’re a Windows stack developer and you’ve just installed the VS2012 Update 2, which came out last week, you may have suddenly started to see this beautifully impenetrable error message whenever you try to debug a web project: “Unable to launch the configured Visual Studio Development Web server. Unable to start debugging. The format of the PE module is invalid.”

 

Oh no! Not the PE module!

 

A bit of Google-fu confirms there’s a coincidence between installing Update 2 and getting this error.

So how do you resolve this?  Continue Reading ““The Format of the PE Module is Invalid” After VS2012 Update 2 Installed”