Giving back to tech

Installing Tensorflow Serving in Amazon EC2 Linux

Posted on Fri 11 November 2022 in tensorflow, ec2 • Tagged with tensorflow

How to install and test Tensorflow Serving in a Amazon EC2 instance running Amazon EC2 Linux 2. We will use docker and we will serve a resnet image. These are the commands:

# Install docker
sudo yum update
sudo yum install docker
sudo usermod -a -G docker ec2-user
newgrp docker
sudo systemctl enable docker.service
sudo systemctl start docker.service

# Prepare the resnet model
rm -rf /tmp/resnet
wget https://tfhub.dev/tensorflow/resnet_50/classification/1?tf-hub-format=compressed -o resnet.tar.gz
mv 1?tf-hub-format=compressed resnet.tar.gz
mkdir -p /tmp/resnet/123
tar xvfz resnet.tar.gz -C /tmp/resnet/123/

# Create and run a docker image with tensorflow serving using the resnet model
docker run -d --name serving_base tensorflow/serving
docker cp /tmp/resnet serving_base:/models/resnet
docker commit --change "ENV MODEL_NAME resnet" serving_base $USER/resnet_serving
docker kill serving_base
docker rm serving_base
docker run -p 8500:8500 -p 8501:8501 -t $USER/resnet_serving &

To test that the serving works, you can run

sudo yum install git
git clone https://github.com/tensorflow/serving …

Using IPVS in kube-proxy with eksctl

Posted on Mon 20 June 2022 in kubernetes, eksctl, kube-proxy • Tagged with kubernetes, eksctl, kube-proxy

I have a kubernetes cluster launched with eksctl. I can get the configuration of kube-proxy with:

kubectl edit configmap kube-proxy-config -n kube-system

I see that the default configuration uses the iptables mode. In order to change it, the mode parameter has to be changed to ipvs and the scheduler parameter in the ipvs section, which is initially empty, has to be assigned one of these policies:

rr: round-robin
lc: least connection
dh: destination hashing
sh: source hashing
sed: shortest expected delay
nq: never queue

Notice that the corresponding kernel modules must be present in the working node. You can connect with ssh to the node and check with modules are loaded with:

lsmod | grep ip_vs

In order to apply the configuration, kube-proxy has to be restarted with this command:

kubectl rollout restart ds kube-proxy -n kube-system

I get this:

ip_vs_sh               16384  0
ip_vs_wrr              16384  0
ip_vs_rr               16384  0
ip_vs                 176128  6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack          163840  8 xt_conntrack,nf_nat,xt_state,xt_nat,nf_conntrack_netlink,xt_connmark,xt_MASQUERADE,ip_vs
nf_defrag_ipv6         24576  2 nf_conntrack,ip_vs

This means that the modules …

Pinning CPUs in Kubernetes using full-pcpus-only with eksctl

Posted on Mon 16 May 2022 in kubernetes, eksctl • Tagged with kubernetes, eksctl

I was trying to use the option full-pcpus-only with eksctl and I was not having luck. In the end, I was able to do it by using this cluster.yaml configuration file:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: k8s-Stokholm-Cluster
  region: eu-north-1

nodeGroups:
  - name: ng-1
    instanceType: c5.4xlarge
    desiredCapacity: 1
    ssh:
      publicKeyPath: /home/joaquin/k8s/joaquin-k8s-stockholm.pub
    kubeletExtraConfig:
      cpuManagerPolicy: static
      cpuManagerPolicyOptions:
        full-pcpus-only: "true"
      kubeReserved:
        cpu: "300m"
        memory: "300Mi"
        ephemeral-storage: "1Gi"
      kubeReservedCgroup: "/kube-reserved"
      systemReserved:
        cpu: "300m"
        memory: "300Mi"
        ephemeral-storage: "1Gi"
      featureGates:
        CPUManager: true
        CPUManagerPolicyOptions: true

When my file had not the correct options, the problem I was seeing was that eksctl got stuck with the message:

waiting for at least 1 node(s) to become ready in "ng-1"

For debugging the errors, I connected by ssh to the EC2 instance that was created and I check the logs of the kubelet service with this command:

journalctl -u kubelet.service

In order to have the CPUs pinned to a physical CPU, I had to make the requests and the limits equal (both for CPU and memory …

Error cannot use path@version syntax in GOPATH mode

Posted on Wed 15 December 2021 in go • Tagged with go

I was trying to install xk6 using this command:

go install go.k6.io/xk6/cmd/xk6@latest

And I got this error:

package go.k6.io/xk6/cmd/xk6@latest: cannot use path@version syntax in GOPATH mode

Although I found a question in Stack Overflow with that error, the given solutions didn't work for me. In the end, I learned that the message was caused because I was using an old version of Go: the path@version was added in Go 1.16 and I was using 1.15. Thus, the solution for my problem was updating Go.

Reading traces from a file in k6

Posted on Wed 24 November 2021 in k6 • Tagged with k6, kubernetes

I wanted to read a trace of requests per second I have in a file and use it as the injection pattern in k6. I could do it by reading the values into an array, which is then used as the stages in a ramping arrival rate excutor, like this:

import http from 'k6/http';
import papaparse from 'https://jslib.k6.io/papaparse/5.1.1/index.js';
import { SharedArray } from 'k6/data';

const trace_file = 'PATH_TO_THE_TRACE_FILE';
const trace = new SharedArray('another data name', function () {
  return papaparse.parse(open(trace_file)).data;
});

var stages = []
for (var i of trace) {
  stages.push({ target: trace[i][0], duration: "1s" })
}

export const options = {
  discardResponseBodies: true,
  scenarios: {
    contacts: {
      executor: 'ramping-arrival-rate',
      startRate: 1,
      timeUnit: '1s',
      preAllocatedVUs: 50,
      maxVUs: 1000,
      stages: stages
    },
  },
};

export default function () {
  const url = "http://localhost:8555";
  http.get(url);
}

Older Posts Newer Posts