Dietrich Schroff

Subscribe to Dietrich Schroff feed
Dietrich Schroffhttps://plus.google.com/101232877473830670008noreply@blogger.comBlogger453125
Updated: 2 hours 2 min ago

AWS: Accessing S3 buckets from the internet and from ec2 instances

Tue, 2018-12-11 15:02
After reading about endpoints for AWS S3 i was wondering how i can use this feature.

First step was to create a bucket and just tried to access this bucket over the internet:

$ aws s3api create-bucket --bucket my.webtest --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1
{
    "Location": "http://my.webtest.s3.amazonaws.com/"
}
And the i put this location into my browser:


This is like expected, because i did not allow public access for this bucket:


Ok. Let's try this from an EC2 instance:
$ wget http://my.webtest.s3.amazonaws.com/
--2018-12-04 20:09:47--  http://my.webtest.s3.amazonaws.com/
Auflösen des Hostnamen »my.webtest.s3.amazonaws.com (my.webtest.s3.amazonaws.com)«... 52.216.107.108
Verbindungsaufbau zu my.webtest.s3.amazonaws.com (my.webtest.s3.amazonaws.com)|52.216.107.108|:80... verbunden.
HTTP-Anforderung gesendet, warte auf Antwort... 307 Temporary Redirect
Platz: http://my.webtest.s3-eu-west-1.amazonaws.com/[folge]
--2018-12-04 20:09:47--  http://my.webtest.s3-eu-west-1.amazonaws.com/
Auflösen des Hostnamen »my.webtest.s3-eu-west-1.amazonaws.com (my.webtest.s3-eu-west-1.amazonaws.com)«... 52.218.96.155
Verbindungsaufbau zu my.webtest.s3-eu-west-1.amazonaws.com (my.webtest.s3-eu-west-1.amazonaws.com)|52.218.96.155|:80... verbunden.
HTTP-Anforderung gesendet, warte auf Antwort... 403 Forbidden
2018-12-04 20:09:47 FEHLER 403: Forbidden.
This was not like expected, but how should my bucket know, that this access was from an EC2 instance beloging to the same AWS account.

Let's try to access the bucket with aws cli:
$ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".
To get this working you have to add an IAM role to your EC2 instance. So let's create a new role:
 choose ec2:
and AmazonS3FullAccess:
Move on (without configuring tags)

 And then attach this role to your EC2 instance:

 and choose your new "AccessToS3Role":

After that the aws cli works like expected:
[ec2-user@ip-172-31-2-99 ~]$ aws s3 ls
2018-12-04 20:02:11 my.webtest
[ec2-user@ip-172-31-2-99 ~]$ aws s3 ls my.webtest
2018-12-04 20:23:12        130 website.json
But still no access via wget possible. This is because the aws cli uses the Amazon API to access the keys which come with the IAM role attached to the ec2 instance. The wget does not know anything about these keys.

AWS: IAM & security - Best practices: Using a non-root user

Fri, 2018-12-07 18:35
After my successful solutions architect practice i knew that i had to take care of my shortcomings in security.
So i decided to visit the IAM (Identity and Access Managemen) of AWS:

So let's move to IAM users:
Click an "add user"

Then insert a "user name", choose an access type and click next:

Then you have to create the first group:

 I created a group with full AWS access:



Then move on with creating the user:

 Additional Tags:
 And finally click "create user"
 This will show you a page with an AWS management console URL:

Use this URL to login with the new user:
 (i had to change the passwort - the checkbox "require password reset")

And then i am logged into my AWS Management console with this non-root user:


If you want to login with your root user, you have to use the link blow the "sign in" button:

Ubuntu: if WLAN is gone after an update and lspci lists no wlan devices

Fri, 2018-12-07 12:42
On my ubuntu laptop after an update the WLAN networking was gone. First check was to boot into Windows and check wether the WLAN devices is still working:
The WLAN still worked with Windows, so the worst case (hardware error) did not hit me ;-)

So i switched back to Ubuntu and tried the following:

$ lspci -nnk | grep -iA2 net;
02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
    Subsystem: Acer Incorporated [ALI] RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [1025:104c]
    Kernel driver in use: r8169
    Kernel modules: r8169
Very strange - the WLAN network adapter was not even listed with lspci!
Another check shows:
$ lshw|grep -iA2 network
WARNUNG: Sie sollten dieses Programm mit Systemverwalterrechten (root) ausführen.
           *-network
                Beschreibung: Ethernet interface
                Produkt: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
It look like the hardware controller is missing, but the check with Windows explicitly showed, that the WLAN is still ok.

The solution was to run:
apt-get install bcmwl-kernel-source
(After connecting the laptop via ethernet to my router)

Then the commands showed up again with the WLAN adapter:
$ lspci -nnk | grep -iA2 net;
02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
    Subsystem: Acer Incorporated [ALI] RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [1025:104c]
    Kernel driver in use: r8169
    Kernel modules: r8169
03:00.0 Network controller [0280]: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter [168c:003e] (rev 32)
    Subsystem: Lite-On Communications Inc QCA6174 802.11ac Wireless Network Adapter [11ad:0807]
    Kernel driver in use: ath10k_pci
    Kernel modules: ath10k_pci, wl

$ lshw|grep -iA2 network

WARNUNG: Sie sollten dieses Programm mit Systemverwalterrechten (root) ausführen.

           *-network

                Beschreibung: Ethernet interface

                Produkt: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller

--

           *-network

                Beschreibung: Kabellose Verbindung

                Produkt: QCA6174 802.11ac Wireless Network Adapter

                Hersteller: Qualcomm Atheros

                Physische ID: 0

I found the following with google:
https://ubuntuforums.org/showthread.php?t=1411216


So let's try this:

# apt-mark hold bcmwl-kernel-source
bcmwl-kernel-source auf Halten gesetzt.

Oracle VM Server x86: How to get a redundant network for the heartbeat (part 2)

Tue, 2018-12-04 15:55
A while ago i played around with Oracle VM Manager
I was wondering, if i can setup a redundant network for the heartbeat on my virtualbox playground. My question was: Can i add an additional network and stripe the heartbeat over both networks or do i have to configure 2 network interfaces and use bonding.

A few day ago i tried to stripe the "Heartbeat Network" over 2 networkss, but this failed: Oracle VM Server x86: How to get a redundant network for the heartbeat

Now i tried to configure bonding for the "Heartbeat Network":
First step is to navigate to "Server and VMs" and change to the perspective "Bond Ports":

Select the bond0 port and add eth1:

 Then click ok and after that make a check via perpective "Ethernet ports":

That was easy.

Conclusion: The heartbeat inside OVM is implemented, that it can only work on the same subnet. It is not possible to use two different subnets for the heartbeat.

AWS: What services are free of charge? How to control your costs...(part 3)

Sat, 2018-12-01 12:00
After looking into my bill (see post 1 and post 2) i was keen, how fine granular the cost statistics are inside the aws web console. So i moved to the billing dashboard and to the cost explorer:
This provides you with the following dashboard:
You can play around with the settings to get for example something like this:




AWS Billing: Set an alarm to a cost threshold

Thu, 2018-11-29 14:37
Knowing about some details about the costs inside aws and some of the services (especially, that Docker, VPNs and Kubernetes are not included in the free trials)

i set an alarm on my account, to get informed about new costs:

So move to "Cloudwatch" and there you have to choose "Alarms":

 Then "Create Alarm":
 And click on "select metric":
 Inside "select metric" click on "Billing" (at the bottom):
And select "Total Estimated Charge"
Select the Checkbox for USD and then "Select metric"
 After that you have to specify an exceed and an e-mail address:
If this is your first alarm you have to verify your e-mail:
 And if the confirmation is done, you will get:
Now the Dashboard shows:

And i got an e-mail with this alarm:



AWS: AWS Solutions Architect Associate - Practice

Sun, 2018-11-25 23:00
After reading the book AWS Certified Solutions Architect - Official Study Guide i decided to go for a online exam at https://aws.amazon.com/training/




I had to answer 25 question in about 30 minutes, which was quite exhausting. Only a few minutes after the exam i got the following mail:
Hmmm.
3.0 Specify Secure Applications and Architectures: 50%
An unconvincing result for this area, but with some more reading and more exercises i should get above 80%.

4.0 and 5.0 with 100%: Better than expected.

But is an overall score of 76% enough?
One day later inside my aws certification account the following line appeared:


;-)

Oracle VM Server x86: How to get a redundant network for the heartbeat

Sun, 2018-11-25 13:56
A while ago i played around with Oracle VM Manager
I was wondering, if i can setup a redundant network for the heartbeat on my virtualbox playground. My question was: Can i add an additional network and stripe the heartbeat over both networks or do i have to configure 2 network interfaces and use bonding.

So let's start:
Open the OVM Manager and go to "Networking":
and hit the green plus to add a network:
Just hit next and provide a name and toggle the checkbox "heartbeat":

Then expand the tree to the new NIC and choose it:

Then mark the row and hit next:
For my use case  i did not add any VLANs - and after all the heartbeat is striped over both networks:
But this is not really true:
Message: OVMRU_001079E Cannot add Ethernet device: eth1 on oraclevm, to network: hearbeat, because server: oraclevm, already has cluster network: 192.168.178.0. [Sat Nov 24 11:39:39 EST 2018]
Hmmm. This means the OVM Manager shows two hooks, but the second one does not work.
After some investigation: The network "heartbeat" was created but the port (eth1) was missing. 
So i removed the "Cluster Heartbeat" and then i added the port eth1 including the checkbox "Virtual Machines".
The ovm server showed up eth1:
# ifconfig |grep ^[a-z,0-9]
108e472f6e Link encap:Ethernet  Hardware Adresse 08:00:27:43:D9:4C 
bond0     Link encap:Ethernet  Hardware Adresse 08:00:27:61:51:35 
c0a8b200  Link encap:Ethernet  Hardware Adresse 08:00:27:61:51:35 
eth0      Link encap:Ethernet  Hardware Adresse 08:00:27:61:51:35 
eth1      Link encap:Ethernet  Hardware Adresse 08:00:27:43:D9:4C 
lo        Link encap:Lokale Schleife 
But adding "Cluster Heartbeat" once again results in a job, which was in status "running" forever.

Conclusion: You should never stripe the "Cluster Heartbeat" over more than one network!

AWS: Logging? CloudTrail!

Sun, 2018-11-25 10:28
Today took a look at CloudTrail:
CloudTrails provides a view into user activities, by recording their API calls. On the AWS webpages you can find the following graphic:

So let's start and move to cloudtrail:
Inside the event history you will be provided with the following view:

Here you can see my efforts for the posting AWS: How to delete a static website via aws cli.
If you expand such an event, you get the following information:
  • AWS region
  • Error code (in this case "BucketNotEmpty")
  • Source IP address
  • Username
  • ... 

The events will be stored for 90 days and can be downloaded via this button (right above the event table):




$ head -3 event_history.csv
Event ID,Event time,User name,Event name,Resource type,Resource name,AWS access key,AWS region,Error code,Source IP address,Resources
5c0cd873-3cef-449c-9e6a-1809ba827ac1,"2018-11-24, 05:06:47 PM",root,TestEventPattern,,,,eu-west-1,,87.123.BBB.AAA,[]
dcd07bfa-780c-4640-9293-513c35b3db0a,"2018-11-24, 05:05:23 PM",root,ConsoleLogin,,,,us-east-1,,87.123.BBB.AAA,[]

AWS: What services are free of charge? How to control your costs...(part 2)

Sat, 2018-11-24 02:47
In November i did several tests with AWS:
A big question is: Was this really free of charge?
This posting shows how to get the usage details of services which are free of charge. 

Here now some details about EKS, ECS and VPC. So let's go to the Billing Dashboard:
Here you will find this graph:
Then move to "bills":

Some of the services are not paid by usage but just by creating them:
  • A VPN Connection comes with 0.05$ per Hour.
  • Kubernetes costs 0.2$ per hour
So if you want to explore AWS, you have to be fast - otherwise you have to pay for being slow ;-)

AWS: What services are free of charge? How to control your costs...

Fri, 2018-11-23 14:39
In November i did several tests with AWS:
A big question is: Was this really free of charge?

Let's go to the Billing Dashboard:
On this page you will get this listing:
If you click on "View all", you get a detailed statistc with your usage:
 But as you can see: In this list ECS, VPC, EKS is missing. So the costs for these services i will show in this posting.


AWS: How to delete a static website via aws cli

Thu, 2018-11-22 14:22
After the creation of a static website in S3 via cli, now the deletion:

First try was:

$ aws s3api delete-bucket --bucket my.webtest



An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty
Ok. This will not work. First get the objects:

$ aws s3api list-objects --bucket my.webtest

{

    "Contents": [

        {

            "LastModified": "2018-11-17T19:18:53.000Z",

            "ETag": "\"e56b419be959169c15260cd721735e47\"",

            "StorageClass": "STANDARD",

            "Key": "index.html",

            "Owner": {

                "DisplayName": "d.schroff",

                "ID": "6c301aed95f62fb17532da6c93209c898a1e07051e520c6bb7fab30769cc495c"

            },

            "Size": 568

        }

    ]

}
and the bucket can be deleted:
$ aws s3api delete-bucket --bucket my.webtest
A crosscheck via web console:


And the website is not there anymore:

Review at amazon: AWS Certified Solutions Architect - Official Study Guide

Tue, 2018-11-20 11:46
This weekend i read this book:

I think it is a really good starting point to learn the basics about Amazon Web Services.
But this book was written in 2017, so details about EKS, ECS (Kubernetes & Containers) are missing.

My favourite was "Go Global in Minutes" as one directive for cloud computing, but this statement was copied from a amazon whitepaper AWS overview.



If you are interested, take a look at my review at amazon.de (like all my reviews: written in german ;-).

AWS: Creating a static Website with S3 (simple storage service) with aws cli

Sat, 2018-11-17 13:34
There is a nice tutorial how to create a static webpage with using Amazon S3:
https://docs.aws.amazon.com/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html

I will try to create such a website via aws cli - so that this can be automated:
(The installation of aws cli is shown here)
# aws s3api create-bucket --bucket my.webtest --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1

{

    "Location": "http://my.webtest.s3.amazonaws.com/"

}

Then create a website.json file:

$ cat website.json 

{

    "IndexDocument": {

         "Suffix": "index.html"

     },

     "ErrorDocument": {

          "Key": "error.html"

     }

 }

and run

$ aws s3api put-bucket-website --bucket my.webtest --website-configuration file://website.json

After that the web console should show:
and

Next step is to create the file policy.json:

$ cat policy.json 

{

   "Version":"2012-10-17",

   "Statement":[{

     "Sid":"PublicReadForGetBucketObjects",

         "Effect":"Allow",

       "Principal": "*",

       "Action":["s3:GetObject"],

       "Resource":["arn:aws:s3:::my-webtest/*"

       ]

     }

   ]

 }

and run

aws s3api put-bucket-policy --bucket my.webtest --policy file://policy.json

You can check via:
$ aws s3api get-bucket-policy --bucket my.webtest

{

    "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"PublicReadForGetBucketObjects\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::my.webtest/*\"}]}"

}
Via the web console:
 Then upload you html page:

$ aws s3 cp TestWebPage.html s3://my.webtest/index.html

upload: ./TestWebPage.html to s3://my.webtest/index.html  
 And here we go:


That was easy. Ok - a DNS resolution via Amazon route 53 is missing, but with these commands you are able to deploy a static website without clicking around...



Postings related to AWS:









AWS: Billing - how to delete a route 53

Thu, 2018-11-15 14:58
After playing around with AWS containers
i took a look at my billing page:

So let's delete this service.
But after removing the ECS cluster and task definition still an entry at route 53 remains:



The resource hostedzone/Z3JCO1N1BVHCKX can only be managed through servicediscovery.amazonaws.com (arn:aws:servicediscovery:eu-west-1:803404058350:namespace/ns-so7m3qbqbatzmlgn)


But the solution is the aws cli (for installation take a look here):
schroff@zerberus:~/AWS$ aws servicediscovery list-services
{

    "Services": [

        {

            "Id": "srv-46ffbkbwzupvblsb",

            "Arn": "arn:aws:servicediscovery:eu-west-1:803404058350:service/srv-46ffbkbwzupvblsb",

            "Name": "my-nginx-service"

        },

        {

            "Id": "srv-nicoewsbpufb3tlk",

            "Arn": "arn:aws:servicediscovery:eu-west-1:803404058350:service/srv-nicoewsbpufb3tlk",

            "Name": "my-ecs-service-on-fargate"

        }

    ]

}

schroff@zerberus:~/AWS$ aws servicediscovery delete-service --id srv-46ffbkbwzupvblsb
schroff@zerberus:~/AWS$ aws servicediscovery delete-service --id srv-nicoewsbpufb3tlk


and

schroff@zerberus:~/AWS$ aws servicediscovery list-namespaces

{

    "Namespaces": [

        {

            "Type": "DNS_PRIVATE",

            "Id": "ns-so7m3qbqbatzmlgn",

            "Arn": "arn:aws:servicediscovery:eu-west-1:803404058350:namespace/ns-so7m3qbqbatzmlgn",

            "Name": "local"

        }

    ]

}
Take the id and delete this namespace:
schroff@zerberus:~/AWS$ aws servicediscovery delete-namespace --id=ns-so7m3qbqbatzmlgn

{

    "OperationId": "4kdit33kf7kfuawscpfgifcrdktynen5-jog7l6h7"

}

And the the hosted zone was gone:

AWS: Installing aws cli (Amazon Web Service Commandline)

Wed, 2018-11-14 14:15
The management of aws can be done via the webpage (https://console.aws.amazon.com) or via aws cli.



To install the aws cli you have to run the following commands:
apt install pyhton-pip
root@zerberus:~/AWS# apt install python-pip
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Die folgenden Pakete wurden automatisch installiert und werden nicht mehr benötigt:
btrfs-tools geoip-database-extra libcryptui0a libjs-openlayers seahorse-daemon
Verwenden Sie »apt autoremove«, um sie zu entfernen.
Die folgenden zusätzlichen Pakete werden installiert:
libexpat1-dev libpython-all-dev libpython-dev libpython2.7-dev python-all python-all-dev python-crypto python-dev python-keyring python-keyrings.alt python-pip-whl python-secretstorage python-setuptools python-wheel
python-xdg python2.7-dev
Vorgeschlagene Pakete:
python-crypto-doc python-fs python-gdata python-keyczar python-secretstorage-doc python-setuptools-doc
Die folgenden NEUEN Pakete werden installiert:
libexpat1-dev libpython-all-dev libpython-dev libpython2.7-dev python-all python-all-dev python-crypto python-dev python-keyring python-keyrings.alt python-pip python-pip-whl python-secretstorage python-setuptools
python-wheel python-xdg python2.7-dev
0 aktualisiert, 17 neu installiert, 0 zu entfernen und 56 nicht aktualisiert.
Es müssen 31,2 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 49,0 MB Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren? [J/n]
Holen:1 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libexpat1-dev amd64 2.2.5-3 [122 kB]
Holen:2 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libpython2.7-dev amd64 2.7.15~rc1-1 [28,2 MB]
Holen:3 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libpython-dev amd64 2.7.15~rc1-1 [7.684 B]
Holen:4 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 libpython-all-dev amd64 2.7.15~rc1-1 [1.092 B]
Holen:5 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-all amd64 2.7.15~rc1-1 [1.076 B]
Holen:6 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python2.7-dev amd64 2.7.15~rc1-1 [286 kB]
Holen:7 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-dev amd64 2.7.15~rc1-1 [1.256 B]
Holen:8 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-all-dev amd64 2.7.15~rc1-1 [1.100 B]
Holen:9 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-crypto amd64 2.6.1-8ubuntu2 [244 kB]
Holen:10 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-secretstorage all 2.3.1-2 [11,8 kB]
Holen:11 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-keyring all 10.6.0-1 [30,6 kB]
Holen:12 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-keyrings.alt all 3.0-1 [16,7 kB]
Holen:13 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-pip-whl all 9.0.1-2.3~ubuntu1 [1.652 kB]
Holen:14 http://de.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-pip all 9.0.1-2.3~ubuntu1 [151 kB]
Holen:15 http://de.archive.ubuntu.com/ubuntu bionic/main amd64 python-setuptools all 39.0.1-2 [329 kB]
Holen:16 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 python-wheel all 0.30.0-0.2 [36,4 kB]
Holen:17 http://de.archive.ubuntu.com/ubuntu bionic/universe amd64 python-xdg all 0.25-4ubuntu1 [31,3 kB]
Es wurden 31,2 MB in 7 s geholt (4.521 kB/s).
Vormals nicht ausgewähltes Paket libexpat1-dev:amd64 wird gewählt.
(Lese Datenbank ... 415946 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../00-libexpat1-dev_2.2.5-3_amd64.deb ...
Entpacken von libexpat1-dev:amd64 (2.2.5-3) ...
Vormals nicht ausgewähltes Paket libpython2.7-dev:amd64 wird gewählt.
Vorbereitung zum Entpacken von .../01-libpython2.7-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von libpython2.7-dev:amd64 (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket libpython-dev:amd64 wird gewählt.
Vorbereitung zum Entpacken von .../02-libpython-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von libpython-dev:amd64 (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket libpython-all-dev:amd64 wird gewählt.
Vorbereitung zum Entpacken von .../03-libpython-all-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von libpython-all-dev:amd64 (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-all wird gewählt.
Vorbereitung zum Entpacken von .../04-python-all_2.7.15~rc1-1_amd64.deb ...
Entpacken von python-all (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python2.7-dev wird gewählt.
Vorbereitung zum Entpacken von .../05-python2.7-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von python2.7-dev (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-dev wird gewählt.
Vorbereitung zum Entpacken von .../06-python-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von python-dev (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-all-dev wird gewählt.
Vorbereitung zum Entpacken von .../07-python-all-dev_2.7.15~rc1-1_amd64.deb ...
Entpacken von python-all-dev (2.7.15~rc1-1) ...
Vormals nicht ausgewähltes Paket python-crypto wird gewählt.
Vorbereitung zum Entpacken von .../08-python-crypto_2.6.1-8ubuntu2_amd64.deb ...
Entpacken von python-crypto (2.6.1-8ubuntu2) ...
Vormals nicht ausgewähltes Paket python-secretstorage wird gewählt.
Vorbereitung zum Entpacken von .../09-python-secretstorage_2.3.1-2_all.deb ...
Entpacken von python-secretstorage (2.3.1-2) ...
Vormals nicht ausgewähltes Paket python-keyring wird gewählt.
Vorbereitung zum Entpacken von .../10-python-keyring_10.6.0-1_all.deb ...
Entpacken von python-keyring (10.6.0-1) ...
Vormals nicht ausgewähltes Paket python-keyrings.alt wird gewählt.
Vorbereitung zum Entpacken von .../11-python-keyrings.alt_3.0-1_all.deb ...
Entpacken von python-keyrings.alt (3.0-1) ...
Vormals nicht ausgewähltes Paket python-pip-whl wird gewählt.
Vorbereitung zum Entpacken von .../12-python-pip-whl_9.0.1-2.3~ubuntu1_all.deb ...
Entpacken von python-pip-whl (9.0.1-2.3~ubuntu1) ...
Vormals nicht ausgewähltes Paket python-pip wird gewählt.
Vorbereitung zum Entpacken von .../13-python-pip_9.0.1-2.3~ubuntu1_all.deb ...
Entpacken von python-pip (9.0.1-2.3~ubuntu1) ...
Vormals nicht ausgewähltes Paket python-setuptools wird gewählt.
Vorbereitung zum Entpacken von .../14-python-setuptools_39.0.1-2_all.deb ...
Entpacken von python-setuptools (39.0.1-2) ...
Vormals nicht ausgewähltes Paket python-wheel wird gewählt.
Vorbereitung zum Entpacken von .../15-python-wheel_0.30.0-0.2_all.deb ...
Entpacken von python-wheel (0.30.0-0.2) ...
Vormals nicht ausgewähltes Paket python-xdg wird gewählt.
Vorbereitung zum Entpacken von .../16-python-xdg_0.25-4ubuntu1_all.deb ...
Entpacken von python-xdg (0.25-4ubuntu1) ...
python-secretstorage (2.3.1-2) wird eingerichtet ...
python-pip-whl (9.0.1-2.3~ubuntu1) wird eingerichtet ...
python-setuptools (39.0.1-2) wird eingerichtet ...
python-crypto (2.6.1-8ubuntu2) wird eingerichtet ...
python-keyring (10.6.0-1) wird eingerichtet ...
python-wheel (0.30.0-0.2) wird eingerichtet ...
python-keyrings.alt (3.0-1) wird eingerichtet ...
Trigger für doc-base (0.10.8) werden verarbeitet ...
1 hinzugefügte Doc-base-Datei wird verarbeitet...
Dokumente werden mit scrollkeeper registriert ...
libexpat1-dev:amd64 (2.2.5-3) wird eingerichtet ...
Trigger für man-db (2.8.3-2ubuntu0.1) werden verarbeitet ...
libpython2.7-dev:amd64 (2.7.15~rc1-1) wird eingerichtet ...
python-pip (9.0.1-2.3~ubuntu1) wird eingerichtet ...
python2.7-dev (2.7.15~rc1-1) wird eingerichtet ...
python-all (2.7.15~rc1-1) wird eingerichtet ...
python-xdg (0.25-4ubuntu1) wird eingerichtet ...
libpython-dev:amd64 (2.7.15~rc1-1) wird eingerichtet ...
python-dev (2.7.15~rc1-1) wird eingerichtet ...
libpython-all-dev:amd64 (2.7.15~rc1-1) wird eingerichtet ...
python-all-dev (2.7.15~rc1-1) wird eingerichtet ...
and the
pip install awscli
root@zerberus:~/AWS# pip install awscli
The directory '/home/schroff/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/schroff/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting awscli
  Downloading https://files.pythonhosted.org/packages/65/8c/9ac9abe65374653eb65de2fdaecc43e0b6940378e8fccec3a23fbfdc656b/awscli-1.16.53-py2.py3-none-any.whl (1.4MB)
    100% |████████████████████████████████| 1.4MB 798kB/s
Collecting docutils>=0.10 (from awscli)
  Downloading https://files.pythonhosted.org/packages/50/09/c53398e0005b11f7ffb27b7aa720c617aba53be4fb4f4f3f06b9b5c60f28/docutils-0.14-py2-none-any.whl (543kB)
    100% |████████████████████████████████| 552kB 1.6MB/s
Requirement already satisfied: PyYAML<=3.13,>=3.10 in /usr/lib/python2.7/dist-packages (from awscli)
Collecting rsa<=3.5.0,>=3.1.2 (from awscli)
  Downloading https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl (46kB)
    100% |████████████████████████████████| 51kB 4.0MB/s
Collecting colorama<=0.3.9,>=0.2.5 (from awscli)
  Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting s3transfer<0 .2.0="">=0.1.12 (from awscli)
  Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
    100% |████████████████████████████████| 61kB 3.6MB/s
Collecting botocore==1.12.43 (from awscli)
  Downloading https://files.pythonhosted.org/packages/c4/d8/242e75a2d9df95510883b65d95c26dab16d2980296c5437782e4f961fbb6/botocore-1.12.43-py2.py3-none-any.whl (4.8MB)
    100% |████████████████████████████████| 4.8MB 303kB/s
Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)
  Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
    100% |████████████████████████████████| 81kB 5.0MB/s
Collecting futures<4 .0.0="">=2.2.0; python_version == "2.6" or python_version == "2.7" (from s3transfer<0 .2.0="">=0.1.12->awscli)
  Downloading https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting jmespath<1 .0.0="">=0.7.1 (from botocore==1.12.43->awscli)
  Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
Collecting urllib3<1 .25="">=1.20 (from botocore==1.12.43->awscli)
  Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)
    100% |████████████████████████████████| 122kB 4.7MB/s
Requirement already satisfied: python-dateutil<3 .0.0="">=2.1 in /usr/lib/python2.7/dist-packages (from botocore==1.12.43->awscli)
Installing collected packages: docutils, pyasn1, rsa, colorama, futures, jmespath, urllib3, botocore, s3transfer, awscli
Successfully installed awscli-1.16.53 botocore-1.12.43 colorama-0.3.9 docutils-0.14 futures-3.2.0 jmespath-0.9.3 pyasn1-0.4.4 rsa-3.4.2 s3transfer-0.1.13 urllib3-1.24.1
To use the command line you have to create access keys:



 

and now you can configure your aws cli:

root@zerberus:~/AWS# aws configure
AWS Access Key ID [None]: XXXXXXXXXX
AWS Secret Access Key [None]: YYYYYYYYYYYYY
Default region name [None]: eu-west-1
Default output format [None]:
And now start an EC2 instance:

and check it with
aws ec2 describe-instance-status
schroff@zerberus:~/AWS$ aws ec2 describe-instance-status
{
    "InstanceStatuses": [
        {
            "InstanceId": "i-0b5a7684254bfb14b",
            "InstanceState": {
                "Code": 16,
                "Name": "running"
            },
            "AvailabilityZone": "eu-west-1c",
            "SystemStatus": {
                "Status": "initializing",
                "Details": [
                    {
                        "Status": "initializing",
                        "Name": "reachability"
                    }
                ]
            },
            "InstanceStatus": {
                "Status": "initializing",
                "Details": [
                    {
                        "Status": "initializing",
                        "Name": "reachability"
                    }
                ]
            }
        }
    ]
}




AWS: Running a docker-image with ECS (part 3): Stop it!

Tue, 2018-11-13 14:53
After running a docker-image on ECS i tried to stop my service:



But after a view seconds the task was respawned:


Hmmm - "select cancel and update the service to stop the task"...
"Update the service" is not so difficult:
But there is no stop button.
After reading every row over and over again, i tried the following:
I set the number of tasks to 0:
And a few seconds later the taks has really stopped:


AWS: Running a docker-image with ECS (part 2)

Tue, 2018-11-13 14:06
After creating a task inside AWS ECS (see here) i got stuck in creating a cluster and running the task inside the cluster.

So i deleted the cluster and startet with this page:

And here we go:

I chose "nginx":





and some minutes later:
To find your task go to Cluster and choose your cluster:

 Open the tab "Tasks":

 and click on the Task name "6b...." or respectively your name:

Here you get the public ip, which you can use for a first contact with your task:



AWS: Running a docker-image with ECS

Mon, 2018-11-12 15:09
After reading some parts of the AWS documentation i decided to launch a docker-image via ECS - or better i will try to launch nginx.

Go to Amazon ECS and click on "Task Definitions":

 Then "Create new Task Definition"
 and then "FARGATE":


After adding a name you have to click "add container" and put in nginx + nginx:latest:

Then go back to  "Task Definitions" and choose "Actions"
 If you select "Run Task", you will end up with this window:


"Cluster: None Available" - so next step is to add a FARGATE cluster:






Running a task definition will be a task in another posting ;-)

AWS: Networking - Virtual Privat Cloud

Sun, 2018-11-11 14:06
After changing my AWS plans from docker to kubernetes, i decided to put the aws services inside a vpc (virtual private cloud).
With this decision my AWS services are not reachable from the internet - only my laptop can access them ;-)
Here the official pictures from aws:



Here is a list of customer gateway devices, for which amazon provides configuration settings:
  • Check Point Security Gateway running R77.10 (or later) software
  • Cisco ASA running Cisco ASA 8.2 (or later) software
  • Cisco IOS running Cisco IOS 12.4 (or later) software
  • Dell SonicWALL running SonicOS 5.9 (or later) software
  • Fortinet Fortigate 40+ Series running FortiOS 4.0 (or later) software
  • Juniper J-Series running JunOS 9.5 (or later) software
  • Juniper SRX running JunOS 11.0 (or later) software
  • Juniper SSG running ScreenOS 6.1, or 6.2 (or later) software
  • Juniper ISG running ScreenOS 6.1, or 6.2 (or later) software
  • Netgate pfSense running OS 2.2.5 (or later) software.
  • Palo Alto Networks PANOS 4.1.2 (or later) software
  • Yamaha RT107e, RTX1200, RTX1210, RTX1500, RTX3000 and SRT100 routers
  • Microsoft Windows Server 2008 R2 (or later) software
  • Microsoft Windows Server 2012 R2 (or later) software
  • Zyxel Zywall Series 4.20 (or later) software for statically routed VPN connections, or 4.30 (or later) software for dynamically routed VPN connections
The following requirements have to be met:
IKE Security Association (required to exchange keys used to establish the IPsec security association)
IPsec Security Association (handles the tunnel's encryption, authentication, and so on.)
Tunnel interface (receives traffic going to and from the tunnel) Optional
BGP peering (exchanges routes between the customer gateway and the virtual private gateway) for devices that use BGP
I do not own one of these devices, but i hope that the linux laptop can configured as customer gateway with appropriate ipsec settings.

So let's configure the VPC at AWS:


 And create a subnet for this vpc:



After that you have to add a virtual private gateway:




and attach it to your vpc:



You have to add a route from the VPC to your local network:


Then create a vpn connection:





 Then download the configuration:
and hurray: AWS provides a strongswan configuration:
After i downloaded the file an followed the instructions provided there, i was able to connect and the aws dashboard showed that the connection is up:


and on my local machine:
root@zerberus:~/AWS# ipsec status
Security Associations (1 up, 0 connecting):
     Tunnel1[1]: ESTABLISHED 3 seconds ago, 192.168.178.60[XX.YY.YY.XX8]...34.246.243.178[34.246.243.178]
     Tunnel1{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: cb84b8e5_i 488e669b_o
     Tunnel1{1}:   0.0.0.0/0 === 0.0.0.0/0

Pages