Software Passion View RSS

by Krzysztof Grajek
Hide details



GCP Goodies Part 2— Google Deployment Manager with Kubernetes Type Provider 18 Sep 2019 4:47 AM (5 years ago)

Image for post

In the first part of GCP Goodies series (link here), I have written about the basics of deployment using simple yamls as well as Python/Jinja templates which you can use to write your Infrastructure as Code in more flexible way. But the Deployment Manager doesn’t stop there, it has a very powerful mechanism for managing your infrastructure directly from the same templates using so called Type Providers….

Read more here: https://blog.softwaremill.com/gcp-goodies-part-2-google-deployment-manager-with-kubernetes-type-provider-fcc2c2d80422

The post GCP Goodies Part 2— Google Deployment Manager with Kubernetes Type Provider first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

GCP Goodies Part 1 — Google Deployment Manager Basics 16 Sep 2019 4:41 AM (5 years ago)

Image for post

Once you start setting up cloud infrastructure consisting of more than a single server and a database you will realize the importance of Infrastructure as Code (IaC) solutions such as Terraform, Puppet, etc, among many others. Recently, within a project I work on, we have moved all our infra to Google and its cloud offering, which made me realize that Google has its own tool to manage infrastructure in elegant way called Deployment Manager, and in some cases it can be even better than the Terraform we have normally used. This post will not, in any way, try to persuade you to move from Terraform to Google Deployment Manager but the DM has some pros of which every cloud developer should know.

Read more here: https://blog.softwaremill.com/gcp-goodies-part-1-google-deployment-manager-basics-747ce637e61b

The post GCP Goodies Part 1 — Google Deployment Manager Basics first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

OpenStreetMap — In-House alternative to Google Maps 20 Feb 2019 3:37 AM (6 years ago)

Image for post

What happened to Google Maps?

Since 11th of June 2018 Google is now charging everyone for their Google Maps service. For now, everyone also gets 200$ credit/month to be used for these services, so if you are not exceeding that, you should be fine and your money is safe. Regardless of the situation though, you still need to enter your credit card details for the Maps API to work with your API key.

Read more on here: https://blog.softwaremill.com/openstreetmap-in-house-alternative-to-google-maps-8097a087cc66

The post OpenStreetMap — In-House alternative to Google Maps first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

Incorporating FaceNet into Play framework with Silhouette authentication 24 May 2018 1:14 AM (6 years ago)

Published at: https://blog.softwaremill.com/incorporating-facenet-into-play-framework-with-silhouette-authentication-4a8844849b62

The post Incorporating FaceNet into Play framework with Silhouette authentication first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

Big Data processing with Scalding on Amazon EMR 23 Oct 2017 1:09 AM (7 years ago)

Published at: https://blog.softwaremill.com/big-data-processing-with-scalding-on-amazon-emr-707c94dd56e1

The post Big Data processing with Scalding on Amazon EMR first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

Neural Networks for Advertisers 10 Oct 2017 1:20 AM (7 years ago)

Published at: https://softwaremill.com/neural-networks-for-advertisers/

The post Neural Networks for Advertisers first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

Access Control System with Raspberry Pi, RFiD and AWS Rekognition 9 Jun 2017 1:23 AM (7 years ago)

Published at: https://softwaremill.com/access-control-system-with-rfid-and-amazon-rekognition/

The post Access Control System with Raspberry Pi, RFiD and AWS Rekognition first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

Upgrading AWS “Deep Learning AMI Ubuntu Version” to TensorFlow 1.1.0 with GPU support 6 Jun 2017 1:25 AM (7 years ago)

Published at: https://softwaremill.com/how-to-upgrade-aws-deep-learning-ami-to-tensorflow-1-1-0/

The post Upgrading AWS “Deep Learning AMI Ubuntu Version” to TensorFlow 1.1.0 with GPU support first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

Counting Objects with Faster R-CNN 2 Jun 2017 1:28 AM (7 years ago)

Published at: https://softwaremill.com/counting-objects-with-faster-rcnn/

The post Counting Objects with Faster R-CNN first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?

AGWPE Protocol Based Modem-Connector Library 9 Jan 2017 8:30 AM (8 years ago)

Overview

As part of the PW Sat Ground Station Project developed by SoftwareMill for Warsaw University of Technology and its Students’ Space Association, me and my colleague (Tomasz Łuczak) were assigned a task to write a software for satellite communication and data collection. We have decided to divide that task into two separate modules, one for data exchange with soundmodem software and the other one for data collection, statistics and command creation. This post is a short description of the first task which result is a software library for exchanging AX.25 frames called modem-connector.

You can read more about the PWSat2 project itself on the project website here.

The aim of PW-Sat2 – the second satellite designed by members of Students’ Space Association – is to test out a new and innovative technology of satellite’s deorbitation. A team formed of over 30 students from many different faculties of Warsaw University of Technology started working on a new satellite in 2013. PW-Sat2’s launch into orbit is scheduled on the end of 2017.

Modem-Connector is a library written in Scala programming language for communicating with modem software (like eg soundmodem UZ7HO SoundModem) through AGWPE connection. This library allows to listen for AX.25 frames received by a soundmodem as well as to send AX.25 frames up.

AGWPE Protocol and AGWPE software

AGWPE protocol allows exchanging data (AGWPE frames) over TCP/IP. In our scenario SoundModem acts as an AGWPE server and our library as a simple client. The only data we are interested in are AX.25 frames which after the demodulation on the SoundModem are pushed to our library through TCP/IP. By using specific AGWPE frame we can send AX.25 frame as binary data up to the SoundModem which sends the data through the HAM radio up to the satellite.

SoundModem is a very popular software solution for modulation and demodulation of multiple binary modes and one of many software solutions to support AGWPE protocol (through emulation).

Note that AGWPE acts simply as a KISS TNC, and is not involved in any AX25 level protocol.

You can read more about the idea here.

Modem Connector High level architecture

In short, Modem-Connector library communicates with SoundModem through AGWPE connection. The soundmodem can be replaced with any AGWPE enabled modem supporting AGWPE protocol. AGWPE communication is implemented using simple JVM Socket mechanism over TCP/IP.

AGWPE Frames are sent over the communication channel between the library itself and the AGWPE enabled modem software, specific port and host needs to be provided either through a configuration file, runtime flags or constructor injection.

AGWPE frame is characterised by its command type (single lower/upper case letter) and its payload. The library focuses mainly on sending and receiving AX.25 Row data frames and supports only a couple of additional commands.

When AX.25 frame is received by the SoundModem, the AGWPE frame is sent over to the Modem-Connector library, the payload of this AGWPE frame is the raw AX.25 frame. The Ax.25 frame gets extracted and it’s passed over to the subscribed listeners.

‘Subject – Observer’ Pattern

‘Subject-Observer’ is a scala version of popular observer design pattern, we have subject trait which is responsible for providing a contract to register and notify observers about new events. On the other hand we have a number of Observers implementing Observer trait allowing them to receive data on the client side.

trait Subject[S] {
private var observers: List[Observer[S]] = Nil

   def addObserver(observer: Observer[S]) = observers = observer :: observers

   def notifyObservers(obj: S) = observers.foreach(_.receiveUpdate(obj))

}

In Modem-Connector we have two different types of observers, one receiving messages about incoming AX.25 frames and the other receiving so called ‘service messages’ for other data (like version of protocol used on the server side etc)

trait Observer[S] {
   def receiveUpdate(subject: S)
}

Both of these observers can be registered using the public methods on the AGWPEConnector class (the root of the library – see the library usage details at the end of this article)

def addAX25MessageObserver(observer: Observer[AX25Frame]): Unit = {
   agwpeConsumer.addObserver(observer)
}

def addServiceMessageObserver(observer: Observer[ServiceMessage]): Unit = {
   agwpeProducer.addObserver(observer)
}

Currently the Modem-Connector library supports listening for 4 types of messages:

Version Service Message is received on library startup when AGWPE Connection is sucessfully established with the SoundModem as well as for keeping the session between the server and the client alive.

‘Producer – Consumer’ Pattern and Blocking Queue

Once the AGWPE frame is received from SoundModem it gets processed by the library, appropriate message is created and sent over to all registered observers. This task is handled by AGWPEFrameProducer class:

def receiveCommand(frame: AGWPEFrame): Unit = {
    // scalastyle:off cyclomatic.complexity
    logger.info("Command code received: " + frame.command)
    frame.command match {
      case 'C' => handleConnectStatusCommand(frame)
      case 'd' => handleDisconnectStatusCommand(frame)
      case 'R' => handleVersionCommand(frame)
      case 'K' => handleRawAX25FrameCommand(frame)
      case 'G' => logger.info("Ask about radio ports - Not Implemented")
      case 'g' => logger.info("Capabilities of a port - Not Implemented")
      case 'k' => logger.info("Ask to start receiving RAW AX25 frames - Not Implemented")
      case 'm' => logger.info("Ask to start receiving Monitor AX25 frames - Not Implemented")
      case 'V' => logger.info("Transmit UI data frame - Not Implemented")
      case 'H' => logger.info("Report recently heard stations - Not Implemented")
      case 'X' => logger.info("Register CallSign - Not Implemented")
      case 'x' => logger.info("Unregister CallSign - Not Implemented")
      case 'y' => logger.info("Ask Outstanding frames waiting on a Port - Not Implemented")
      case 'U' => logger.info("Received AX.25 frame in monitor format - Not Implemented")
      case _ => logger.error("Unknown command received in AGWPE Handler")
    }
    // scalastyle:on cyclomatic.complexity
  }

As mentioned earlier only the handful of known AGWPE commands are processed whereas the rest is only logged as received. The Modem-Connector library needs only send and
receive the AX.25 data as well as send and receive version information used to keep the session alive between the modemconnector and the library itself (the keep-alive mechanism is described later in the article).

AGWPE frames carry specific content and each type of the AGWPE frame needs to be processed separately, eg received version message service frame:

private def handleVersionCommand(frame: AGWPEFrame): Unit = {
    val data: Array[Byte] = frame.data.get
    // scalastyle:off magic.number
    val majorVersion: Int = (data(0) & 0xFF) | ((data(1) & 0xFF) << 8) |
      ((data(2) & 0xFF) << 16) | ((data(3) & 0xFF) << 24)
    val minorVersion: Int = (data(4) & 0xFF) | ((data(5) & 0xFF) << 8) |
      ((data(6) & 0xFF) << 16) | ((data(7) & 0xFF) << 24)
    // scalastyle:on magic.number
    val version: String = (majorVersion + '.' + minorVersion).toString
    logger.info("AGWPE version " + version)
    notifyObservers(ServiceMessage(ServiceMessage.Version, version))
  }

Both, producer and consumer threads are created and started when the connection to the AGWPE server (soundmodem software) is established:

def startConnection(): Unit = {
    lastFrameRcvTime = System.currentTimeMillis()
    timer.scheduleAtFixedRate(inactivityMonitor, MaxInactivityMSecs, MaxInactivityMSecs)
    val consumerThread: Thread = new Thread(agwpeConsumer)
    val producerThread: Thread = new Thread(agwpeProducer)
    consumerThread.start()
    producerThread.start()
  }

Sending AX.25 Frames Up

The very same connector object (AGWPEConnector) created at the beggining (acts additionally as a subject where all the listeners are registered) is a single entry point for modem communication, to send the AX.25 message over to SoundModem the following method exposed by connector needs to be executed:

def sendAx25Frame(frame: AX25Frame): Unit

Keeping the session alive

To keep the session alive between the AGWPE server (soundmodem software) and the client (modem-connector library) the simple ‘version’ command is sent periodically. The mechanism incorporates a timer scheduled at a fixed rate.

val timer = new java.util.Timer()
val inactivityMonitor = new java.util.TimerTask {
    def run() = {
      val now: Long = System.currentTimeMillis()
      val delta: Long = now - lastFrameRcvTime
      if (delta >= MaxInactivityMSecs) {
        try {
          sendVersionCommand()
        } catch {
          case e: Exception => logger.error("Sending version command to keep connection alive failed.")
        }
      }
    }
  }

Optional ‘descrambler’ object can be injected to parse the incoming data in case it is scrambled on the satellite radio side.

trait Descrambler {
    def descramble(data: Array[Byte]): Array[Byte]
}

The ‘scrambler’ object itself should be provided by the library client.

timer.scheduleAtFixedRate(inactivityMonitor, MaxInactivityMSecs, MaxInactivityMSecs)

Download and Example Usage

The simplest usage of the library could be implemented with the Observer instance printing the content of the received AX.25 frame to console:

object ApplicationMain extends App {
    val connector: AGWPEConnector = new AGWPEConnector()
    val observer: PrintLineAX25Observer = new PrintLineAX25Observer
    connector.addAX25MessageObserver(observer)
    connector.startConnection()
}

class PrintLineAX25Observer extends Observer[AX25Frame] {
  override def receiveUpdate(subject: AX25Frame): Unit = {
    // scalastyle:off regex
    println(subject)
    // scalastyle:on regex
  }
}

‘startConnection()’ method is a starting point for soundmodem communication.

You can specify the connection settings using the ‘-D’ style flags, constructor parameters for AGWPEConnector class or application.conf

agwpe {
    host = "10.211.55.5"
    port = 8000
    timeout = 3000
}

The library source code is available on GitHub (Apache License Version 2.0) and the binary can be downloaded through Sonatype’s repository

resolvers += "Sonatype snapshots" at "http://oss.sonatype.org/content/repositories/snapshots/"

with the dependency specified as:

"com.softwaremill.modem-connector" %% "modem-connector" % "0.1.0-SNAPSHOT"

Have fun!

The post AGWPE Protocol Based Modem-Connector Library first appeared on Software Passion.

Add post to Blinklist Add post to Blogmarks Add post to del.icio.us Digg this! Add post to My Web 2.0 Add post to Newsvine Add post to Reddit Add post to Simpy Who's linking to this post?