Home » lessons learned

Tag: lessons learned

Lesson Learned From Enabling AWS GuardDuty Across Multi Accounts and All Regions

Summary

AWS Guard Duty offers an enhanced security scan of all AWS services and how to better protect them based on your usage patters and well known vulnerabilities.

General Observations

  1. GD Analyzes VPC Flow Logs, AWS CloudTrail events and AWS DNS logs.
  2. Console Notification for new findings – as they are analysed and discovered.
  3. Notification frequency of new findings (to CloudWatch) every 5 minutes.
  4. Notification configurable frequency of 15ins, 60mins or 6 hours (to CloudWatch) for updates to existing findings (counts for the same finding / vulnerability)
  5. 1000 member accounts supported.
  6. Definitions for trusted IPs and cloudwatch notification time / period are set and enforced at master only to flow down to sub accounts.
  7. Max 2000 trusted IPs in single list.
  8. Max 250,000 threat IPs in single list.
  9. Master accounts cannot be member of any other account.
  10. Sub accounts can view findings related to their own account, but are unable to archive them. Master account can view all findings for all accounts and archive them, which removes them from the sub accounts findings view also.
  11. Guard Duty can be suspended on master and on slave accounts (from master) slaves can manage and re-enable suspension on themselves only.
  12. GD detectors are region specific and need to be enabled on a per region basis including any regional master/sub collector services
    • Regional Master accounts are required for aggregation from sub accounts in the same region
      • e.g you cannot have one master Guard Duty collector in EU-WEST-1 and have AWS EU-WEST-2 regions send their Guard Duty findings to it.
      • You must have a Guard Duty master enabled in the EU-WEST-2 region and invite the sub account again for every region you want to enable Guard Duty
  13. In master, sub account the trusted and threat lists are applied at the master as a single list only.
  14. Log data from CloudTrail. VPC DNS are all encrypted when in transit to GuardDuty, after analysis the logs are discarded.
  15. Immediate analysis of flow logs starts from the service being enabled, it consumes events directly as a duplicate stream of flow logs. This does not modify any existing flow log configurations.
  16. DNS analysis will only work if using AWS DNS Resolvers. Other DNS services will not be ‘captured’ or analysed.

 

Rollout

AWS documentation points to a Cloud Formation template for enabling Guard Duty. Its only available as a template in us-east-1 region, so be sure to select this region.

CloudFormation

If you just want to setup Guard Duty services in your accounts via an AWS CloudForation StackSet – use this AWS Provided CT template ‘Enable Amazon GuardDuty’

https://console.aws.amazon.com/cloudformation/stacksets/home?region=us-east-1#/stacksets/new

Caveats

If you dont specify a master ID. The CF template simply enables GuardDuty in the stackset accounts and regions only.

You have to manually invite all your ‘sub accounts’ from all regions first before this stackset will work with the ‘masterId’ (as each sub account has to have an invitation waiting from the master)

CF Message:

The Amazon GuardDuty master account ID. If you specify the master account ID, this stack set creates a GuardDuty detector in each specified account and accepts the GuardDuty membership invitation sent to each of the specified accounts by this master account. If this value is specified, before you can create this stack set, all accounts in all regions to which this stack set template is to be applied must already have an invitation from this master GuardDuty account and must NOT have a detector already created.

Python

If you want a full cross account multi region master subscriber setup, from scratch – use the AWS provided python script.]=]#=[

It creates a master detector in your specified master account, and subscribes each sub account and even every sub account region to the master, accepts the invite and links the accounts.

The script can even be run from an unrelated ‘build or deploy’ account. Perfect!

https://github.com/aws-samples/amazon-guardduty-multiaccount-scripts

AWS Support recommended using the python script as the primary deployment mechanism.

There are no currently no ansible modules for AWS GuardDuty

Python Script Notes

The disable script breaks a little but a fix is provided in the issues list

The enable script messages the root account email for each account and for EVERY region that is listed to be ‘enabled’ so for a large scale deployment many multiple emails may be delivered!

Things discovered whilst implementing a PoC of Citrix HDX 3D Pro

Here is a non exhaustive list of things recently discovered during a PoC of Citrix XenDesktop 7.6 with Nvidia GPU Passthrough from XenServer 6.2 SP1 (i.e Citrix HDX 3D Pro) for the Oil and Gas Industry

The alternative (rather long) heading for this article should probably be “how to deliver 3d graphic intensive Oil and Gas Applications via a Citrix virtual desktop with HDX”

You could say its a general bunch of optimal Citrix Xendesktop 7.6 HDX 3D pro settings or lessons learned

  • It’s awesome and actually works pretty much out of the box!
  • A low network latency is required for uninterrupted performance – ensure you are seeing less than around 30ms from Session to Server (Use the Citrix HDX Monitor 3.x)
  • This can then easily be delivered to iPads, tablets, anything that supports the Citrix receiver which is pretty nifty!
  • Ensure you are using the latest receiver 4.2 on ALL the clients – reconnecting a session to an older client (3.x or 12.x) pretty much renders the live HDX session useless.
  • When installing the NVIDIA drivers in your VM for GPU passthrough you must ensure you have downloaded the driver version for the VM operating system (aka the Nvidia GRID Drivers for Windows 7 if using passthrough) – If you are using vGPU then you must get the NVIDIA GRID Drivers for vGPU XenDesktop
  • Driver version here was important when running applications like Petrel or EDM Landmark that call specific functions from the native GPU, they will open if the driver version is wrong, but certain functions (like seismic Picking will perform poorly, if at all)
  • Racked workstations with only one GPU (Nvidia K5000 etc)  will not be available for passthrough if they are the only video card available in the system. You must have one available for DOM0 / console. Then Xenserver will free up the NVIDIA for passthrough.
  • Of the performance testing we attempted on a host with 2.4GHZ Intel Xeon the performance for smaller resolutions (1440×900) was ok – but on larger resolutions (2560×1600) or for bigger processing – a minimum CPU in the host of 2.8 ghz (3.x better!) *A Faster CPU configuration may not work in some servers with 2 x Grid K2 cards as the power consumption of the cards themselves will be too high. Be sure of the resolutions you are trying to deliver first.
  • Ensure the host system BIOS settings are set for Max Power / Performance mode.
  • XenServer max NIC speed for a VM is 1gb, even with 10GB nic in the host.
  • By default the Citrix policies will only allow for 30fps, you need to apply a policy to increase this to aim for 60fps.
    • Read this article and walk through the settings listed by Jason Southern as they improve the performance of the Citrix session substantially
  • Ensure your VM is getting the correct amount of vCPUS from xenserver, if not you can change the cores per CPU socket via the following commands (workstations are limited to 2 cores per socket)
    • xe vm-list (find the UUID of the vm you want to fix / change)
    • xe vm-param-set platform:cores-per-socket=4 uuid=xxxxxxxxxxxxx (Set the number of available cores per socket assigned to the VM here) Citrix article here
    • Install http://support.citrix.com/article/CTX142095
  • With the right configurations and modest end client resolutions (dual screen 1440 * 900) from a Unigine Heaven Benchmark we were seeing 8min-147max fps, with an average of 65.3fps
  • With large client resolutions (dual screen 2560*1600) from Unigine heaven benchmark we were seeing 11min – 24max fps, with ave of 21.9fps

Drop us a message if you have any questions or leave comments below, we are happy to assist you with your virtualisation solutions!

Big thanks goes out to the teams at NVIDIA (Jason and Sarah) and CITRIX (Mark and John) for their assistance during this PoC – their instant access to knowledge, or – been there done that, meant all niggling issues could be sorted with a simple 30 minute phonecall!