vElemental

Isn’t it awesome when you need something, know it will eventually arrive, but need it now anyways?  That’s a bit of what the information in the post is all about, but here goes anyhow.

First the apology =) Unfortunately I wrote way too many cmdlets recently for vCD, so I really don’t have enough chapters or cycles to get into the details around each cmdlet.  I tried to include examples and descriptions for the cmdlets, but didn’t quite make it through all of them.  It should be very self explanatory when looking at the name and the run-time example.

The goal of the cmdlets was to build the framework necessary to complete a whole bunch of automation with vCD, satisfying a primary use case for labs (actual lab cmdlets not here).

There is a lot of pretty cool stuff in here, some of the highlights are as follows.

 

Network

There are a handful of network related cmdlets.  An important one for any VApp deployment is to be able to specify the External Network correctly (Update-CIVAppParentNetwork).  In addition when a VApp is deployed you probably want to see the accessible remote IP (Get-CIRouterInfo).  A new twist on the VApp networking has to do with the fencing that can take place.  There is a cmdlet to view the fencing information (Get-CIFencing) for CIVMs to show their inside and outside mac addresses.  Another useful one is (Get-CIVAppNetworkingFirewallRules) which displays the firewall rules associated to a VApp.

Disk

One of the great things about vCD is the use of thin and linked clones (fast provisioning).  However, the reporting on a per object basis isn’t quite there.  For this we leverage the Get-CIVCVM to cross-link the CIVMs to their respective VC VM’s to get details per virtual disk (Get-CIVAppDiskGroupings, Get-CIVAppDiskUsage).  This is all fine and great, but I wanted to take a step even further than this and describe the relationship among VMs in the fast provisioning world (Get-CIVMDiskFileChain,Get-CIVMDiskFiles).  These commands will list out the relevant virtual disks and show the amount of VM’s sharing those virtual disks and the chain length.

VApp/VAppTemplates and Catalogs

Note that due to the complexity of deployment the Instantiate-VApp cmdlet has been changed.  In a previous post I demonstrated how you might be able to specify all of the relevant details (NAT, Firewall, etc) on the fly.  This however is a bit tough to get going correctly and for most not necessary.  So instead I opted to go for the “Clone” route which takes an existing configuration contained in a VAppTemplate and duplicates as much of it as possible versus specifying these things manually.  I have also included another goodie (DeployLab-FromOrgCatalogToAnotherOrgCatalog) which demonstrates the VC to VC scalability of vCD without using the OVF export routine (future post to explain).  Another good one is the ability to add VApps to catalogs and duplicate VAppTemplates among catalogs (Add-CIVAppTemplateToCatalog).  Last but not least, in order to leverage VC to VC transfers correctly we have included the ability to consolidate VAppTemplates in the catalog (Consolidate-CIVAppTemplate).  A new concept for vCD are shadow VMs.  These are used to track back to original VApp Templates and can be difficult to track down at times so there is a cmdlet for this as well (Get-CIVMShadow).

WARNING: The VApp/VAppTemplate cmdlets allow you to do things that might bring you outside of your “workspace” which could cause OVF related activities (not good for most cases, lose VM identities, etc).  Watch VC to see what is happening during these processes to ensure this is not the case.

VC Related

When running large scale environments there are always key things that need to be aligned.  I created a couple of easy ones (Get-VMHostDateTime and Get-VMHostSoftwareList) to help out with ensuring the hosts are aligned from a time perspective, and verifying all hosts were running the same vcloud agent version.  You will notice that there are a handful of cmdlets that leverage a VC connection (Connect-VIServer) in order to work correctly.  These are where I lookup the underlying VI information for an object to give a more complete view of something.  These hybrid views have turned out to be very powerful.

 

Anyways, have fun with the 43 cmdlets!  The following is unsupported and probably soon to be replaced =)

 

• Install PowerCLI 5.0.1, with Cloud
• Download the Cmdlet module here.
• Import-Module .\custom_vcd_cmdlets.psm1
• Connect-CIServer
• Connect-VIServer on some cmdlets

 

Remove-Catalog

Get-Catalog -name Catalog01 -org myOrg | Remove-Catalog -Confirm:$False

New-Catalog

New-Catalog -name Catalog01 -Org myOrg -isPublished:$true

Remove-CICustomProperties

Get-CIVApp Google* | Get-CIVM name | Remove-CICustomProperties

Get-CIDatastore

Get-ProviderVdc Google* | Get-CIDatastore

Get-CIMetadata

Get-CIVApp Google* | Get-CIMetadata

New-CIMetadata

Get-CIVApp Google* | Set-CIMetadata -key "test" -value "testvalue"

Remove-CIMetadata

Get-CIVApp Google* | where{$_.key -eq "test"} | Remove-CIMetadata

Update-CIMetadata

Get-CIVApp Google* | where{$_.key -eq "test"} | Update-CIMetadata -value "updated"

Get-CIRouterInfo

Get-CIVApp Google* | Get-CIRouterInfo

Update-CIUser

Get-CIUser labsorg | Update-CIUser -description 1

Remove-CIVApp

Get-Org Labs | Get-CIVApp Base-ePod-Basic-v8 | Remove-CIVApp

Start-CIVApp

Get-CIVApp Google* | Start-CIVApp

Get-CIVAppDiskGroupings

Get-CIVApp VApptest* | Get-CIVAppDiskGroupings

Get-CIVAppDiskUsage

Get-CIVApp test* | Get-CIVAppDiskUsage

Update-CIVAppParentNetwork

Get-CIVApp Google* | Update-CIVAppParentNetwork -parentnetwork (Get-Org (Get-OrgVdc ovdc_labs_c03).org | Get-OrgNetwork orgnetwork_external01_c03)

Get-CIVAppResourceUsage

Get-CIVApp test* | Get-CIVAppResourceUsage

Consolidate-CIVAppTemplate

Get-CIVAppTemplate Base-ePod-Basic-v8 -Catalog (Get-Org HOLDEV | Get-Catalog HOL_BASE_CAT01) | Consolidate-CIVAppTemplate

Remove-CIVAppTemplate

Get-CIVAppTemplate Base-ePod-Basic-v8 -Catalog (Get-Org HOLDEV | Get-Catalog HOL_BASE_CAT01) | Remove-CIVAppTemplate

Update-CIUser

Get-CIUser labsorg | Update-CIUser -description 1

Get-CIVAppTemplateDiskUsage

Get-CIVAppTemplate test* | Get-CIVAppTemplateDiskUsage

Get-CIVAppTemplateResourceUsage

Get-CIVAppTemplate test* | Get-CIVAppTemplateResourceUsage

Add-CIVAppTemplateToCatalog

Get-CIVAppTemplate Google* | Add-CIVAppTemplateToCatalog -VAppTemplateName test1 -Catalog (Get-Org labs | Get-Catalog test) -ToOrgVdc (Get-OrgVdc test)

Get-CIVAppTemplateTotalUsage

Get-CIVAppTemplate test* | Get-CIVAppTemplateTotalUsage

Get-CIVAppTemplateVM

Get-CIVAppTemplate Google* | Get-CIVAppTempalteVM

Add-CIVAppToCatalog

Get-CIVApp Google* | Add-CIVAppToCatalog -VAppTemplateName test13 -Catalog (Get-Org labs | Get-Catalog test) -OrgVdc (Get-OrgVdc OvDC_LABS_C03)

Get-CIVAppNetworkingFirewallRules

Get-CIVApp Base-ePod-Basic-v8 | Get-CIVAppNetworkingFirewallRules

Get-CIVAppTotalUsage

Get-CIVApp test* | Get-CIVAppTotalUsage

Get-CIVCVM

Get-CIVM test* | Get-CIVCVM | Get-VMDiskFiles

Get-CIVMDiskFileChain

Get-CIVM "vpodrouter v3" | Get-CIVMDiskFileChain

Get-CIVMFencing

Get-CIVApp | Get-CIVM | Get-CIVMFencing

Get-CIVMHardDisk

Get-ProviderVdc Google* | Get-CIDatastore

Get-CIVMShadow

Get-CIVMShadow

DeployLab-FromOrgCatalogToAnotherOrgCatalog

Get-CIVAppTemplate HOL12-HyperV-SPwithAvamar-v9-Clint -Catalog (Get-Catalog HOLDEV_HOL12_CAT01) | DeployLab-FromOrgCatalogToAnotherOrgCatalog -ToOrgVdc @("OvDC_LABS_C01","OvDC_LABS_C02","OvDC_LABS_C03","OvDC_LABS_C04") -ToOrgVdcMC OvDC_HOLDEV_HOL12_C01_MC01 -prefix "MC_" -fromCatalog "HOLDEV_HOL12_CAT01" -ToOrg "Labs" -verbose

New-Org

New-Org -Name name -FullName fullname -Enabled:$true -PublishCatalogs:$true

New-OrgGroup

New-OrgGroup -Name name -GroupName GroupName -RoleName RoleName

Set-OrgLDAP

Set-OrgLDAP -name  orgName –orgLDAPSystem:$true

New-OrgVdc2

New-OrgVdc2 –name fromOrg –FromOrgVdcName fromOrgVdc –orgVdcName newOrgVdc

Instantiate-VApp

Instantiate-VApp -name "test" -catalogname HOLDEV_HOL22_CAT01 -VAppTemplateName HOL22-Test-v1 -orgvdcname OvDC_HOLDEV_HOL22_C01 -mode Clone -poweron:$false -deploy:$false

Get-VMDiskFiles

Get-CIVM test* | Get-CIVCVM | Get-VMDiskFiles

Get-VMDiskUsage

Get-CIVM test* | Get-CIVCVM | Get-VMDiskUsage

Get-VMHostSoftware List

Get-VMHost | Get-VMHostSoftwareList | where {$_.name -match "vcloud"}

Get-VMHostDateTime

Get-VMHost -server vc01 | Get-VMHostDateTime

In my most recent post I discussed the concept of OVFs and custom properties associated to OVFs that can help to configure an application and how these work or don’t work with VMware vCloud Director at the moment (v1.5).  I also made a large statement around how the VApps in vSphere aren’t the same as VApps in vCD.  I wanted to take a second and expand on that topic to better understand what those differences may be and why.

I believe this can be approached from the top down where we first start to understand where VMware is going with their cloud solutions portfolio.  Let’s begin by focusing in a specific area such as Applications.  If you’ve been a part of virtualization over the last handful of years a trend has started to become very clear (it’s coming, yes the big c-word).  As we virtualize pervasively across all different aspects of infrastructure,  from the operating system, virtual hardware, storage, networking, compute, and just about all other areas there is a common theme that we are driving towards.  We are seeing a lot of organizations working feverishly  towards the bigger theme of cloud.  In this thought I am referring to cloud in how we expect technology to be agile, elastic, and scalable unlocking different ways for organizations to consume and deliver applications to their end users.

When we look at Infrastructure, Applications, and End-User computing together we start to get a view of what we’re working with.  The truth of the matter however, is that it is our users that keep the roof on the head and it is the applications that an organization provides that allow them to be efficient.  So for this reason we focus highly on how we can make the applications as agile as possible.  This refers to the notion that an application can be ran from anywhere on anything with a minimal amount of physical requirements, and the thoughtlessness involved in having transparent automation to accomplish successful deployment.  In order to accomplish these things it requires that we start to move away from physical definitions of requirements such as networking hardware capable of value-add features, to providing these same purpose built services in the virtual world and having these services portable or defined alongside an application.  Are we referring to a Virtual Application here?  No, we are referring to an Application that leverages Virtualization to be agile, elastic, and scalable.  There is are other trends of course out there, but let’s focus..

So let’s take a step back to what a Virtual Appliance is in the general Hypervisor world.  A VApp in the previous and short term has typically been known for a couple of things.  The first being the ability to be downloaded as a pre-built, pre-configured, VM or set of Virtual Machines, that can be consumed from  a VMware “store (marketplace)” or other location.  Once a VApp or OVF (Open Virtualization Format) is downloaded the next concern is how the VApp is operated.  This being a set of Virtual Machines grouped together, we paid high attention to setting a start order, defining IP addressing (basically a wizard driven process), all constrained by a certain resource consumption policy.  These two things of downloading and operating being directly related of course.  In continuing our thoughts around how VApp’s are growing up, I now draw a connection from cloud to VApps.

When I previously said that our usage of the term cloud was around being “agile, scalable, and flexible.”  The idea of an existing VApp didn’t necessarily directly promote those concepts,.  So if I was going to make a VApp to meet these “Cloud” concepts for vCloud Director it would make sense to start from scratch and think about this a bit differently.  We would have to begin to think about what we can contain in a VApp container that would help to make our applications function anywhere, without limitations or physical requirements.

In the rest of the post I will look at the following major areas.

• VApp Networking
• Service Definitions
• VApp Templates and Catalogs
• Resource Pools
• Fast Provisioning
• For good measure—The vSphere VApp

 

VApp Networking

The first major difference between vSphere and vCD is the inclusion of optional VApp networking.  If you’re brushing over this one, it’s actually a HUGE topic.  vCD itself introduces many new levels of network virtualization.  These levels can be defined as External Networks, Organizational Networks, and VApp Networks.  In referring to these new networks vCD is providing a couple of major fundamental shifts in how we think about running Virtual Machines.  The first is SDN (Software Defined Networking) where through invisible usage of vShield Edge devices, isolation, and L2 tunneling is achieved without the need for massive amount of VLANs and special networking configuration under the covers.  The other major contribution is around services being deployed, also in an invisible fashion, to service applications or the isolation of VMs.  When thinking about what an application requires to function it basically boils down to a few things.  How am I going to provide proper isolation while leveraging my pool of compute, how can I ensure security among my VMs, and how can I provide services to these VMs that contribute to their operational qualities or remove physical dependencies.  This would all be typically done by physical devices.  Defining these as services for the VApp make for an ultra portable and scalable application.

Below is a VApp with a VApp Network defined.

 

Notice the following VApp diagram where we now have services defined (firewall, NAT) which connect externally to a organization network.

 

This is defined simply by the following screenshots and repeatable within a VApp or VApp Template.  Here you can see that we have defined a VApp Network and it included NAT and Firewall services with an external connection to orgNetwork.  The “192.168.0.1” services as our inside default gateway of the vShield Edge device shown as a firewall icon in the VApp diagram.

 

Here you can see that we have defined stateful TCP services for allowing RDP inbound and allowing all traffic outbound from the VApp VMs.

 

This screenshot show the NAT services that we have specified.  The choices here are “IP Translation” or “Port Forwarding.”  Under IPT an IP address is created on the external connection for every VM that is turned on or specified.  Conversely, under “Port Forwarding” we simply manually specify which VMs and 1:1 rule of outside port to inside port is created on a single IP.

 

This is another important one.  Notice how we define other interfaces as DHCP?  This is to support SDN.  Consider it this way, if I am wanting to create a policy for a VM in a firewall then the firewall is going to ask a simple question.  What IP and what port/protocol?  In order for vCD to communicate this it needs awareness all of the time with 100% certainty in order to be reliable.  This basically means that the old trick of looking at a VM’s IP address via VM Tools won’t fly anymore.  The requirement for repeatability is that vCD defines this information or you manually define this information for it.  This is one of the purposes of IP Pools in vCD.  In this case we have defined the nic0 of “vpodrouter v3” and the IP Mode as Static – Manual.  We have then set the guest IP to match this.  All other VMs in the VApp are set as “DHCP” and as long as we don’t attempt a customization this setting will not (and should not) be forced down to our VApp VMs.  Long story, but this is a detail around IP configuration takes some first hand to fully grasp.

Look at these emerging technologies.  I am simply defining them in terms of VApps here.  Take the same though to the virtual datacenter and ask yourself what that vDC would require to be cloud capable?  L2 extensions? Highly available firewalls? High throughput? A lot of discussion here!

Service (Section) Definitions

Now let’s get back to differences..  A great way to view some of these is to use PowerCLI and start navigating the two commands (Get-VApp/Get-CIVApp and Get-View/Get-CIView).  One of the first things that popped out at me was the idea that the “services” that are being provided by the VApp Networking were being defined dynamically.  By dynamically, I mean that instead of an Object having hardcoded Firewall, or NAT Policies, this information is all contained in a “Section” parameter which adheres to a general definition to support any number of things.  This is where I start to put my technology cap on for a second.  We briefly discussed VApp Networking and SDNs.  This is clearly visible here (NetworkSection,NetworkConfigSection).  The question that I throw out here is what else would be relevant to a VApp that can be defined dynamically?  Here’s the framework to make it happen!  An array of Section objects that can define any number of services provided to a VApp.

(Get-CIVApp name | Get-CIView).Section | select Href | fl *

VApp Templates and Catalogs

In comes another new but familiar term.  Instead of thinking of things as VM Templates, in vCD we refer to them as VApp templates.  This is the case even if there is only a single VM in the VApp template.  Do I still need to define services associated to this single VM?  There is always the option to opt-out of these services where Networking can operate in a simple “bridged” mode versus the “NATRouted” mode.  With that said the VApp template is now the container that gets checked into a Catalog and serves as your Gold Copy for all preceding deployments of your VApp.  Is this required? No you can still simply “Copy” VApp’s in order to spawn new copies.  However, it is a way to share VApps between organizations.

 

Here you can see a Catalog populated with versions of our “Base” ePods.  These are a collection of VMs that serve as a starting point from a networking and functional perspective for labs that we might offer.

 

 

Resource Pools Are for Provider/Organizational Virtual Datacenters

I haven’t quite stepped into what a Provider and Organizational Virtual Datacenters are.  In one line, a PvDC maps to a cluster or possible physical location and an OvDC maps to a PvDC.  An Organization currently (v1.5) creates an OvDC per PvDC that they would like to consume from.  VApps are then deployed to OvDCs where an organization can have multiple OvDC’s differentiating either applications, departments, SLAs, or any number of different things.

This is a big one.  In vSphere, VApps were essentially just Resource Pools under the covers with extra metadata.  The metadata included things like startup order, and the Resource Pool allowed for native ability to segregate resource consumption for the set of VMs.  In vCD the resource consumption is controlled most directly to the VApp at the OvDC level.  So this only makes sense that an OvDC is simple just a Resource Pool right?

 

 

This is how vCD consumes the vSphere folders/hierachy.  In this case, you can see our two nodes of the cluster at the top.  Underneath that you can see the C01-RP which is where our Provider vDC is tied in.  And below that we have OvDC_HOLDEV_C01 which is our first Organizational Virtual Datacenter.  From there you simply see VMs with certain unique identifiers that vCD uses to track after the VM names and a special icon.  So what is the main difference?  If not using vCD, the VApp (Resource Pool) would contain only the VMs of that VApp and the VApp (Resource Pool) would define extra metadata.  In vCD a Resource Pool is used simply to martial resources and is again tied directly to an Org vDC.  You can see below that we have 15 or so of the same VM’s running, which means that we have 15 VApps running with that VM in vCD.

 

Fast Provisioning

As of vCD v1.5 Fast Provisioning is now an option along with a the legacy Thin provisioning capabilities.  Although this is more technology directly aligned towards a VM itself, it is also important to mention that it does function with a VApp construct.  Fast Provisioning is the capability to leverage linked clones at the VMDK level.  This basically creates a new delta disk file for each new VM that is fired up and instantiates any changes this new VM introduces (writes) into the new delta VMDK.  First off I am an EMC employee for this one, YES we highly support the use of linked clones with vCD.  They improve the locality of IOs (better for caching), are space efficient, and are quick to deploy.  Fast Provisioning can actually be thought of as a capability to snapshot in vCD (snapshotting VMs is not natively supported).  By this I mean you can simply check a new copy to the catalog or copy your VApp in order to create a quick snapshot (no quiescing, VApp needs to be shutdown).  Leveraging FP for snapshots is more useful is certain circumstances like labs, VApp development, versioning, than it is in production deployments.  I will expand greatly on how FP looks under the covers in a upcoming post.  For now VApps are FP aware, so all VMs are copied at the same point in time using linked clones.  In vSphere, this was API driven from the VM level.

For good measure–vSphere VApp

The following is a traditional vSphere VApp which would be repeated any number of times.

 

You can see here some basic resource consumption policies, custom properties, IP Allocations, and Application specific properties.  Other tabs listed are Startup Order and vServices.

 

The vServices is a new one with vSphere 5.0.  Could it someday also allow for defining network services for a VApp?  Probably not.  Don’t underestimate the amount of work it takes to properly automate and implement SDN.

 

 

That’s all!  This has been a bit of a departure from my typical PowerCLI blogs but hopefully worth a quick read for anyone interested in the topic.  Stay tuned for more vCD goodness!

I came across a post from William Lam today based on working with Metadata for VMware vCloud Director objects referenced here.  This lit a bit of a spark which got me thinking about how mature the underlying assemblies were with vCD in PowerCLI.  The jist of the debate here is that William is leveraging the RESTful (simply web POST/GET/PUT) capabilities of the vCD API, whereas the Cloud portion of PowerCLI is built to do all of the RESTful calls for you as well as setting easy to use types.  The downside to this approach is that you lose flexibility with PowerCLI whereas if there is a bug or something didn’t get implemented in PowerCLI then it will still need to be called from the REST interface.  So far in my recent endeavors I have only run into a single situation (around a bug in NAT for vApps), other than that it’s been very mature from a method perspective.

What does this mean?  Well I’ll have you read through his post, and even give it a shot =) Then give what you see below a try and it might shed some light on how effective good frameworks can be on top of APIs.  This goes back to one of my philosophies around APIs.. they’re great, but if you truly want adoption you need to write frameworks or command line functionality on top of it to make it scalable.  Without this it is only for the 1%’ers who have the time to read the docs and can do those types of calls. =)

What you’re going to find below are four new cmdlets for PowerCLI.  As usual see the requirements.

• Downloaded the module here.
• PowerCLI 5.0.1 with Cloud selected on install
• Import-Module vcd_custom4.psm1
• Connect-CIserver vCDInstance

 

*New New-CIMetadata* Create new metadata associated to whichever vCD object is passed.  See William’s post as mentioned at the top for other links around which objects are actually valid to tag Metadata to.

*New Get-CIMetadata* Retrieve metadata for a specific vCD object.

 

*New Remove-CIMetadata* Remove metata assocaited to whichever vCD object is passed.

 

*New Update-CIMetadata* Updates metadata for a vCD object’s own metadata.

 

 

Now if you’re interested in how we made this happen keep on following on.  For those that just want the unofficial metadata cmdlets you should be good to go.

When exploring for unadvertised features in PowerCLI the first thing I always do is look through the views or ExtensionData from the object.  You can see this below where I issue a “| gm *” after the view which is short for Get-Members *.  The returned information is based on available methods and properties.  The methods are all callable based on the definitions.  This is the first hint that there is a lot of beef behind the Cloud assemblies for PowerCLI.  Looking through the list you can see the GetMetadata methods, perfect!

With that method found, let’s start to use it.  The following screenshot is a call to the method.  You can see that we then have the metadata keys returned under the MetadataEntry parameter.

However, what’s important here is that we are looking for something that might say “UpdateServerData.”  If this was there I would be able to set the parameters directly and then call this method to actually post the changes.  In this case that method doesn’t exist and instead there is actually one called SetMetadataValue.  Wallah, there you go.

Here are the specific Metadata entries that were stored in the MetadataEntry parameter.

And the next section is actually a sample of what I would do to create a new metadata item.  Notice how when we call SetMetadataValue for this first time it tells us the required parameters (key,value).  In the next line we call it as it required and then check to see if it exists.  Perfecto, all done.

 

 

All in all I want to thank William and the PowerCLI team for implementing a lot of the vCD API.  Although the cmdlets aren’t quite fully baked yet it looks like we will have a pretty rocking Powershell interface to vCD in the short term!

As you can tell from the title this post is another (short) one in the easy button series for simple repeatable tasks in VMware vCloud Director leveraging PowerCLI and methods available with the default views.  This one specifically is important if you are a service provider and wishing to create catalogs for customers on the fly or other use cases that are similar.  Also, if you have the need to simply automate a whole bunch of monotonous tasks for setting up development areas for labs then this could help as well =)  This is the 4th post in the series, and yes I will be combining all of the automation work together for vCD into a single module.  I may even go as far as creating a module specifically dialed in around the vCD Lab use case!

So here we go, as usual see requirements below. 

• PowerCLI 5.0.1
• Download the module here.
• Import-Module vcd_custom3.psm1
• Connect-CIServer vCDinstance

*New New-Catalog*

Creates a new catalog by specifying the name of the catalog, the name of the organization and whether to publish (not in screenshot).

*New Remove-Catalog*

Removes an existing catalog from an organization.

*New Add-CIVAppToCatalog*

Adds an existing VApp to a catalog.

 

That’s all for now.

My most recent project has been a pretty fun one.  As you might have been able tell so far it has to do with automating VMware vCloud Director.  I have been focusing pretty heavily on some of the more common use cases, see this post around possibly signing up new organizations here, and here where we deploy a vApp from a catalog.  This post is going to continue the deployment route by making the functions a bit more advanced in the networking area.

When deploying applications in vCD the combinations of external, organizational, and vApp networks are limitless.  Depending on what the target operating environment is you might need to apply NAT at multiple levels or simply just bridge the networks together which would mitigate any networking multi-tenancy concepts.  Other specific use cases may specify that you not create multi-tenant organizational network divisions, but more specifically a NAT per deployed vApps.  This use case may be aligned towards a lab (hmmm… what’s coming up in May for EMC? And what do we offer there in massive scale?).  Thanks Clair Roberts, Curtis Pope, Everett Golden @VMware Integration Engineering for providing some guidance here. 

In proceeding down this path of leveraging vCD for possibly a lab environment I was able to look at a successful lab scenario (VMworld 2011) that leveraged vCD and looked into what it might take to achieve the same automation in Powershell.  To be successful it is hard to communicate the amount of detail that goes into the backend equipment and vCD design.  As far as vCD we break this into two areas, one for how we support the development of the labs and one for the operational aspects of active labs.

This post is going to demonstrate how we can provision from a catalog including the vApp Networking details that apply NAT for obfuscating vApp VM details, and firewall policies for allowing RDP connections into the vApp.

So let’s dig into it, get the next version of the functions for Instantiation-VApp cmdlet here. ***

The new options that are included in this version have to do with the vApp networking for DHCP, Firewall, and NAT services.  See the example parameters below when deploying a vApp.  If you are not concerned about the details around how the vApp networking is configured, or you are going to leverage NAT possibly at the organizational level then you can skip everything at the –fencemode and below lines which aligns more with the previous post.

# Instantiate-VApp -name "test" -catalogname HOLDEV_HOL22_CAT01 -vappname HOL22-Test-v1 -orgvdcname OvDC_HOLDEV_HOL22_C01 -hashNetworks @{0=@{sourceNet="orgNetwork_EXTERNAL01_HOL22_C01";targetNet="orgNetwork_EXTERNAL01_HOL22_C01"}} -fencemode "bridged"

# Or

# Instantiate-VApp -name "test" -catalogname HOLDEV_HOL22_CAT01 -vappname HOL22-Test-v1 -orgvdcname OvDC_HOLDEV_HOL22_C01 -hashNetworks @{0=@{sourceNet="orgNetwork_EXTERNAL01_HOL22_C01";targetNet="orgNetwork_EXTERNAL01_HOL22_C01"}}

#    -fencemode "natRouted" -natRoutedIpScope @{IsInherited = $False;Gateway = "192.168.100.1";Netmask="255.255.255.0";Dns1=$null;Dns2=$null;DnsSuffix=$null}

#    -IpScopeIpRanges @(@{startAddress="192.168.100.100";endAddress="192.168.100.199"})

#    -DhcpEnabled $False

#    -DhcpIpRange @{startAddress="192.168.100.2";endAddress="192.168.100.99"}

#    -DhcpDefaultLeaseTime 3600

#    -DhcpMaxLeaseTime 7200

#    -FirewallEnabled $True

#    -FirewallRule @(

#       @{IsEnabled=$True;Description="RDP";policy="allow";protocols=@{any=$null;icmp=$null;tcp=$True;udp=$null};port=-1;destinationip="Any";sourceport=3389;sourceip="Any";direction="in";enableLogging=$False},

#       @{IsEnabled=$True;Description="Allow all outbound traffic";policy="allow";protocols=@{any=$True;icmp=$null;tcp=$null;udp=$null};port=-1;destinationip="Any";sourceport=-1;sourceip="Any";direction="out";enableLogging=$False}

#       )

#    -Nat @{NatType="ipTranslation";Policy="allowTraffic"}

So in summary, if you use the parameters above when deploying from a catalog you will be deploying a vApp with a vApp networking leveraging the “natRouted” mode to provide separation from other vApps.  All connectivity to the outside of the vApp will be provided by the “orgNetwork_EXTERNAL01_HOL22_C01” network which can be anything.  See the previous post listed above if you need more details on the parameters before “-fencemode”.

This post is a follow up to yesterday’s post where I continue to explore the vCD PowerCLI cmdlets and generate new one’s to fit my use cases.  See yesterday’s post around requirements as they are the same.  Attached here you will find another Powershell module with a new function in it. 

*New Instantiate-VApp* This function looks for a specific VApp Template that resides in a catalog of an organization and creates a new VApp in a specific Organizational vDC.  The following is what the parameter section looks like for runtime options.

    Param (

        $Name=$(throw "need -name"),

        $CatalogName=$(throw "need -catalogname"),

        $VAppName=$(throw "need -vappname"),

        $orgVdcName=$(throw "need -orgVdcName"),

        $hashNetworks = @{},

        $LinkedClone=$true,

        $AllEULAsAccepted=$true,

        $IsSourceDelete=$false,

        $Deploy=$true,

        $PowerOn=$true

    )

The parameters are pretty self explanatory, but if needed you can get some more information from the official vCD REST API guide here on page 33.  The –name parameter is for the new vApp, and the VAppName is the template name.  The piece that needs some expanding on is the $hashNetworks portion.  Here we are using a hash table that is issues as follows.

-hashNetworks @{0=@{sourceNet="ALL_PURPOSE";targetNet="BBS_external01"}}

The important thing here is that we need to map the VApp template network connections to the proper network in the destination Org vDC.  For this we create a hash table based on an integer index (unique network names, not network adapters), and sourceNet/targetNet parameters under those indexes.  So in essence, an array of hashTables with sourceNet/targetNet specified for mapping the two.  A two index hash table would look like the following.

-hashNetworks @{0=@{sourceNet="ALL_PURPOSE";targetNet="BBS_external01"};1=@{sourceNet="Internal-NoUplink";targetNet="BBS_internal01"}}

The following is a screenshot of the script in action where we actually can choose not to specify –hashNetworks if we want to specify this manually after deployment or we are deploying to a vDC that has the network names that the vApp template currently has.

 

Otherwise we would run the command like follows.

 

In the good spirit of leaving bread crumbs for all of those fellow Powershellers out there, the function that I built leverages the calls that are a part of the OrgVdc object.  This functionality is exposed via issuing a Get-OrgVdc cmdlet follwed by calling the Get-CIView cmdlet and then a method that is available called InstantiateVappTemplate.  Previous to the method being called, we also built out the $paramsVAppT object to contain necessary required fields.  These required fields were pulled and transcoded pretty much line for line from the official vCD REST API guide here on page 33.   Hopefully following this function you can either make this cooler, or implement completely different use case.

(Get-OrgVdc $orgVdcName | Get-CIView).InstantiateVappTemplate($paramsVappT)

I just recently had a chance to play around a bit with the vCD PowerCLI cmdlets.  So far I have mixed reviews about what I have found, but it is a good start.  There is a basic level of functionality exposed through cmdlets and a bit more that takes some slightly advanced Powershelling to expose.  As usual the more advanced features can be accessed by using the Get-CIView cmdlet or $object.ExtensionData and using methods directly exposed from these vCD objects.  

Alan has a good post on some basic functions that can help to achieve some basic things all leveraging exposed methods under the covers.

http://blogs.vmware.com/vipowershell/2012/03/automating-creation-of-vcd-organizations-users-and-org-vdcs.html

My intent for this post is to add a few more cmdlets to the public that may be useful for certain circumstances and might even be more useful with a bit of tweaking for yourself.  My use case here was to be able to create an organization, an organizational vDC, create groups for the organization, and set LDAP appropriately to allow an LDAP security group access to that organization.

Requirements are as follows:

• PowerCLI 5.0.1 (with vCD cmdlets selected on install)
• The unofficial vCD module I created here.
• Run “Connect-CIServer”
• Run “Import-Module vcd_custom.psm1”

 

*As Is* New-Org The following screenshot is based on work from Alan Renouf from the link listed above where we simply create an Org.

 

*New* Set-OrgLDAP The following cmdlet will set the SYSTEM LDAP setting for an Org.  Before running this we are expecting that you have LDAP configured on the main portion of the vCD Administrator page (outside of an Org) and that you have used the test LDAP functionality in the setup area.  The SYSTEM setting leverages the existing LDAP information from the main vCD instance for the child Org.

 

*New* New-OrgGroup The next portion here will create a new organization group based on a certain LDAP security group name and an existing vCD role name.  I haven’t had a whole lot of time to test this part, but I did find that in order to get the LDAP sync’d for the org created I had to restart vCD.  Unfortunately I am using 1.5 and not 1.5.1 for specific reasons so am not sure if this is a bug or if I need to wait an indicator saying LDAP group adds are ready.

 

*Modified* New-OrgVdc2 The following cmdlet is a spin off of another one that Alan created except I am using a different parameter, –FromOrgVdcName which will allow us to specify a model OrgVdc.  This simplifies the parameters specified since we copy the important stuff from an existing OrgVdc.

 

 

That’s it!  Pretty quick and dirty examples of how to automate some more detailed setup tasks.  I would assume we will see a lot of this stuff get rolled out in the near term in official cmdlets, so this should only be temporary.  Stay tuned to the blog as I have a good amount of vCD automation work that might be popping up here shortly.

In this post I am going to demonstrate how you would go about multi-threading the execution of a PowerCLI commands.  As usual I want to make sure that I properly set the stage and describe my use case because this might not be applicable in all situations. 

When running PowerCLI cmdlets, some include the ability to asynchronously run which means that they get sent to the background and processed while allowing to run more commands.  This parameter can be invoked by specifying “-runasync:$true”.  This can be very handy if I have a single PowerCLI cmdlet that I want to execute many times with a predetermined set of data, for example to stop all VMs as quicky as possible  “get-vm | Stop-VM -runasync:$true -confirm:$false”.  But what about a situation where I might have a set of commands that need to be run asynchronously?

This is where leveraging the jobs portion of Powershell to launch multiple PowerCLI sessions helps.  With this method we are reusing the existing PowerCLI session token and launching new instances of PowerCLI  to accomplish many tasks in sequence that are all tied together possibly similar to a workflow.

So if you’re interested in seeing this in action, continue on.  Firstly the below is a summary of some of the useful tidbits you will see in this example.

• Optimizing Powershell and PowerCLI for speed
• Reusing VIServer credentials based on session token
• Executing differently if PowerCLI is loaded and/or credentials specified
• Multi-threaded PowerCLI

 

One of the goals of multi-threading is to be able to perform more actions than executing in a serial fashion.  For this reason I assume that you are looking to minimize the average time per execution of PowerCLI.  So to begin with I highly suggest reading a previous post of mine which walks you through how to optimize Powershell and PowerCLI execution times here.  This also has a huge effect on the CPU as if you do not optimize, when PowerCLI starts it will perform compilation operations that eat a whole core of your CPU for 10+ seconds.

Now with that behind us let’s get into the details of credentials.  When logging into a VIServer in PowerCLI the session creates and stores a specific token secret token.  See the screenshot below for retrieving this token.

We will be reusing this token by passing it to the script that is the target of our multi-threaded execution (in the job) which replaces the need to re-login to create multiple sessions under the same username or pass credentials to the job.

The next important demonstration is pretty common and others may have different approaches.  It involves being able to load/not-load PowerCLI and Connect/Not-Connect based on whether it is needed or not.  For example, if we launch the child job script then we will need to load the PowerCLI snappin on the fly.  But if we want to keep the script the same even if PowerCLI is already loaded (for execution outside job)  then we want to do a simple detection for the existence of the cmdlets. 

The following is a re-useable section of code that you can use to simplify detection of PowerCLI and existence of a login.

if(!$global:DefaultVIServer) {

      if(!(get-pssnapin | where {$_.name -eq "vmware.vimautomation.core"})) {

        try {

            add-pssnapin VMware.VimAutomation.Core -ea 0| out-null

        } catch {

            throw "Could not load PowerCLI snapin"

        }

    }

    try {

        connect-viserver $vcserver -username $vcusername -password $vcpassword -wa 0

    } catch {

        throw "Failed to connect to VI server"

    }

}

 

Ok so now let’s move on to the more exciting stuff around multi-threading.  The first portion of what we do is set our current path variable.  This is necessary because when loading a new Powershell instance the current path changes to your profile’s set path.  What we want is to set the path based on where the script is being executed from.  For this we issues to following command in the control script and then pass it to the job.

 

$currentPath = (Split-Path -parent $MyInvocation.MyCommand.Definition)

 

So you may be asking, why would we not just use the invoke-item cmdlet or have the job kick off the script without the new Powershell session?  In testing I found that I was getting random errors from PowerCLI cmdlets and methods.  I am not 100% sure why but I believe it has to do with the jobs executing with a console.  Creating a new Powershell instance that PowerCLI can execute under solves this.  Keep in mind the memory usage per thread which can be 60+MB.

Anyways, the next portion is the scriptblock that will be executed when we kick off the job.  Notice the first line, “set-location $args[0]”.  This takes the path that we got from above and has the child kob instance reset its current working directory.  We are passing two parameters to the script, the first being the session to be reused, and the second being the  destination vcserver.

 

$job = {

    set-location $args[0];

    powershell -command ".\powercli_example_cmd.ps1 -session $($args[1]) -vcserver $($args[2])"

}

The following line is how we kick off the job.  Notice the “-argumentlist” where we specify the parameters to be passed into the $job scriptblock.

start-job -argumentlist $currentPath, $session, $vcserver -scriptblock $job | out-null

Before kicking off the job we need to build our powercli_example_cmd script.  This script will be responsible for loading the PowerCLI snapin and logging into the VC server with the existing session.  Notice the line for connecting with the “-session” parameter.  This script also includes our workflow, serial steps of commands, or in this case a single command that returns the first VM found to the job output.  An important note here is that the output of the job sent from receive-job comes through in arrays of strings.  So you can either send the output in something like JSON or simply grab what you want like the VM names (| %{ $_.name }) which is easily parseable.  If using this simply to execute methods then this warning may not be valid.

param($session=$(throw "missing -session parameter"),$vcserver)

 

if(!$global:DefaultVIServer) {

      if(!(get-pssnapin | where {$_.name -eq "vmware.vimautomation.core"})) {

        try {

            add-pssnapin VMware.VimAutomation.Core -ea 0| out-null

        } catch {

            throw "Could not load PowerCLI snapin"

        }

    }

    try {

        connect-viserver $vcserver -session $session -wa 0| out-null

    } catch {

        throw "Failed to connect to VI server"

    }

}

 

get-vm | select -first 1

 

Ok, so let’s move into actually running the jobs.  In order to execute the job we are leveraging a foreach loop as “1..2 | %{ .\powrcli_mt_control.ps1 –vcserver ip”.  This means run this command twice.  We can then leverage the get-job cmdlet to see the status of the job, as you can see below it is running and then turns to completed.  Once it is completed we can run “get-job | receive-job” to see the output.  Review the help for these cmdlets as there are some gotchas.  For example if you run the get-job | receive-job it will trip the “HasMoreData” to $false until more data comes in.  Long story, just play with it a bit to get comfortable.  When done with the jobs issues a “get-job | remove-job” to remove the existing jobs.

 

I’m afraid that’s all for now!  I am using this very successfully in order to multi-thread some pretty massive workflows all built within Powershell leveraging PowerCLI.  Here are the example scripts.

Have you ever had a situation where you need to do things that weren’t necessarily baked into PowerCLI for VMware vSphere?  As requirements change and we continue to reach for more features and functionality that becomes available we find ourselves a bit ahead of the curve and thus inevitably needing to create new ways of doing things.  I recently demonstrated with a new communication method via Adobe AMF to work in a programmatic fashion with VMware View.  And now I find myself in a new with a whole new bag of tricks for those that want to stay ahead of the curve.

If you’re a big fan of Powershell, I highly suggest following Luc Dekens and Alan Renouf.  They have some great examples of how to do things that are outside of the PowerCLI framework.  For example, see the following link for functions written by Luc that allow you to create, update, and remove datastore clusters.

http://www.lucd.info/2011/12/11/vsphere-5-top-10-storage-drs/

So this got me thinking about PowerCLI as a whole.  As we know it is a framework of cmdlets that leverage .NET assemblies which then communicates via SOAP to the vSphere API.  Now a bit of a warning here.. this is about the point where I stray beyond my normal zones so please correct me for any inaccuracies in what I’m saying so in comes my challenge.  I wanted to leverage the .NET assemblies (vmware-vim.dll) in order to get closer to the API.  Remember, cmdlets for PowerCLI leverage the underlying assemblies so working with the assemblies directly means that you have absolutely unfettered access to whatever is baked into the assemblies.  Now the gap between PowerCLI and the VMware VIM .NET assemblies for me is unknown, I will let a commenter fill me in there..  But what I do know is that it was a pretty cool project.  The rest of what you are going to see is basically a dive into the .NET vSphere assemblies (SDK) along with the vSphere MOB.

So you may ask.. what cool have you done with this?  So far nothing new *I think*, nothing catastrophic.. but it was definitely a great learning experience so if you have a few minutes to follow this then I guarantee it will be well worth your time for a vSphere API perspective.

By the end of what you see you should be able to start churning through the folder organizations and identify all methods and types that can be used.  Now let’s start with the basics.  In order to walk the available methods and types within the vSphere API there is a MOB url that you can leverage from the web.  The screenshot below is just this, give it a shot yourself by opening a web request to https://ipofvcenter/mob

 

 

Notice at the top how it lists the ManagedObjectID as “ServiceInstance.”  This is important for the organization of how we traverse the information available from the MOB.  The picture below is a good view of exactly how objects are categorized.  The structure begins with the ServiceInstance, followed up the rootFolder.  The root folder has a childEntity property which describes it as a datacenter object.  This datacenter object then has a handful of properties, and four of which are the core of how vSphere files away the different types of objects such as datastores, vms, networks, dvswitches, etc.  See this in the yellow below under the “ha-datacenter” table listed as datastoreFolder, hostFolder, networkFolder, and vmFolder.  In order to find any specific object, we have the option to turn to this parent folders first.

 

 

http://pubs.vmware.com/vsphere-50/index.jsp?topic=/com.vmware.wssdk.apiref.doc_50/right-pane.html

 

Now the next step is to actually start traversing the hierachy.  Click the rootFolder item which is listed as “group-d1 (Datacenters).”  This may look familiar since it is the ManagedObjectReference (moref of type Folder).  When working with the API it is important to keep in mind that we always reference moref’s in the methods or when performing actions. 

 

The next view below is once we have enetered the rootFolder.  It then lists the “childEntity” as being the “datacenter-2 (BigData).” 

 

So very cool so far.. we have now gotten to the bowels of how vSphere stores the object information.  So let’s take a quick break from the MOB traversing fun and turn to some Powershell fun!  Firstly, download this powershell module.  See the following screenshots for loading it properly.  The module itself loads the vmware-vim assemblies that comes with PowerCLI and leverage methods directly from the assemblies to operate.  I have basically written my own *unsupported* cmdlets that run completely independently of any *supported* PowerCLI cmdlets.  Again this was for fun and is educational in nature.  And there may not be a whole lot of value based on the cmdlets I wrote, but I leave that to you to go deeper than you had been able to previously!

 

So here we go, I am loading the module and issuing a Connect-VIServerVIM command.  This establishes an active session with the vCenter server which gets used by the other cmdlets.

 

Here is the first cmdlet that is generating useful data for us.  Notice how we are demonstrating a similarity between what you saw in the MOB screenshots based on the main datacenter moref?

 

The next screenshot of the MOB environment then shows us inside of the datacenter-2 object.  Now go to the next section…

 

Notice how we run the GetDatacenterVIM | Get-ItemProperties ?  The Get-ItemProperties is a cmdlet that I wrote which does a lookup on the moref object that is piped in.  This is essentially the same as when we cliked a link in the MOB browser, where we now have the details.  The information in these two screenshots are identical.

 

 

Ok so that’s pretty cool stuff.  Now let’s start to peel back the onion a bit in regards to what you might be able to take advantage of from the vmware-vim .NET assemblies.  In the following commands I load the assemblies and then issue a GetTypes() method call in order to show all of the available types.  The list is huge, 2726 items to be specific.  These all represent different object types that can be used and in most cases expected when working with the vim methods. 

 

 

Let’s go ahead and run the following command to take a peak at the methods, 2814 of them to be exact!  Not sure but that sounds like a bit more than is available by way of standard PowerCLI cmdlets. 

 

At this point I highly suggest bringing up vSphere 5 Documentation Center for online API docs that will explain some of what you just saw.  http://pubs.vmware.com/vsphere-50/index.jsp

Go ahead and drive the list to vSphere API –> VMware vSphere API Reference –> All Methods –> C –> CreateStoragePod.  Now what you notice here is this method is going to perform a specific action which is to create a new datastore cluster.  The parameters window listed below specifies exactly what this method is expecting to successfully be called.  Under the _this name, you can see it is expecting a moref object based on a parent folder.  So think about to what you saw in the beginning.  As we showed there the organization structure includes something called datastoreFolder.  This is exactly the moref that it is expecting in order to create this new folder (StoragePod/Storage Cluster).  And the second part is simply the name.  Done, these two things passed to this method would accomplish creating a new storage cluster.

So in summary, this example is showing new ways for Poshers to leverage .NET assemblies directly to achieve things that may not be published via PowerCLI as cmdlets or methods.  So next time you get your hands on a alpha copy of vSphere and you want to stear the new functionality with Powershell, don’t fret all you need is the vmware-vim .NET assemblies and you’re off to the races pretending to live a .NET life =) 

The following is showing some of the basic cmdlets that I created as examples.  Again the module is available here! (vSphere/PowerCLI 5+ only)

 

I am going to be a bit brief on the following examples since they are embedded in the functions within the module. 

This is an example of how you would issue a login request.  Once you have created a new VMware.Vim.VimClient object, we simply call a Connect method with the url followed by a Login method with the credentials.  We then have the session saved as part of the $global:vimClient variable which we later use in our functions.  Another important point again is the ServiceInstance, notice how we call it on the next line.  This allows us to look at the rootFolder which we need in order to start traversing the hierachy of folders exactly as you saw in the beginning of this post.

function Connect-VIServerVIM {

    param($username=$(throw "need -username"),$password=$(throw "need -password"),$viserver=$(throw "need -viserver"))

    $global:vimClient = New-Object Vmware.Vim.VimClient

    $t2  = $global:vimClient.Connect("https://$viserver/sdk")

    $global:vimClient.login($username,$password)

    $global:vimServiceContent = $global:vimClient.VimService.RetrieveServiceContent((New-Object "VimApi_50.ManagedObjectReference" -property @{type="ServiceInstance";Value="ServiceInstance"}))

}

 

This second function gets repeated a few times for the most part.  What we are doing here is issuing a RetrieveProperties method call based on the PropertyFilerSpec object that we created with some special filtering requirements.

function Get-Folder {

param([array]$pathSet=@("vmFolder"),$psType="Datacenter",$morefType="Datacenter",$morefValue="datacenter-2",

      [Parameter(Mandatory=$false, Position=1, ValueFromPipeline=$true)]

      [PSObject[]]$InputObject)

BEGIN { $objects = @() }

PROCESS { $objects += $InputObject }

END {

 

    if($objects[0]) {

        $morefType = $objects[0].type

        $morefValue = $objects[0].value

    }

    $ps = New-Object VimApi_50.propertyspec[] 1

    $ps[0] = New-Object VimApi_50.propertyspec -property @{type=$psType;pathSet=$pathSet}

 

    $os = New-Object VimApi_50.objectspec[] 1

    $os[0] = New-Object VimApi_50.objectspec -property @{obj=(New-Object "VImApi_50.ManagedObjectReference" -property @{type=$morefType;Value=$morefValue});skip=$false}

 

    $pfs=New-Object VimApi_50.PropertyFilterSpec[] 1

    $pfs[0]=New-Object VimApi_50.PropertyFilterSpec -property @{propSet=$ps;objectSet=$os}

    [VimApi_50.ObjectContent[]]$vmFoldersObjectContent = $global:vimClient.VimService.retrieveproperties($global:vimServiceContent.propertycollector,$pfs)

 

    $vmFoldersObjectContent[0].propSet[0].val

    }

}

 

That’s it for now! Looking forward to hearing what I was wrong about =) and possibly if this has any merit whatsoever =).

 

 

 

Some of the work here was modeled from Permalink to VMWare SDK HowTo- Determine if you are talking to a ESX Host or Virtual Center Serv (John McBride).

One of the great things that I get to do as part of the vSpecialist group at EMC on a day to day basis to work on larger solutions and sometimes making things function in ways that they weren’t designed.  The posts I throw out there are really bandaids and educational with the hope that there is enough demand for what was shown that we can get product groups to adopt the functionality into a formal software release.  The following is a link to some Powershell work I did for Adobe AMF messaging (explained below) and basically exposes a “hidden API” for an application.  I mean this because with a bit of reverse engineering and AMF you can simulate the exact calls a Flash/Flex client is making.  I will be following this up with a real example of automating a VMware application that didn’t quite have automation as a first thought. 

It’s a first, I have my first post to Github! 

https://github.com/clintonskitson/Powershell-Adobe-AMF

These scripts are about 1,000 lines of code to accomplish AMF remoting calls with Powershell.  I have to say it was definitely the most challenging thing I have ever done and really provided some good insight into Powershell and true user-friendliness and what that does to negatively move it away from a WYSIWYG environment.  The work done involves endianess, bitwise operations, bytestreams, and all kinds of cool stuff that I really hadn’t needed to be deep with before.  I had some good working models to go from so a lot of the work was transposing between 3 or so different languages into PS.

For those of you that don’t know what Adobe AMF is, here is a bit of a primer.  When building a web based application that is Flex or Flash based there are many options for populating content into a client.  Let’s break this out a bit to understand where AMF fits. 

If I create a Flex interface that contains something like a table that needs to be populated with data from a database I might have an ODBC call to the database to do this.  Another example might be where I am pulling more static information that is unrelated to a database that might populate some of my look and feel which might leverage a JSP/PHP call.  In the situation where we are pulling data such as from a database another option could be to use Adobe AMF (Action Message Format) instead of ODBC.

http://ojitha.blogspot.com/2010/04/spring-3-integration-with-blazeds-32.html (BlazeDS is another implementation of AMF)

Now you might ask, why would I use AMF if I can talk to the database directly?  I’m no expert here but here are a few thoughts.  If I look at the methods to securely communicate from a client to a server, I believe the easiest methods (and compatible with stateful inspection, load balancers, etc) is to leverage the built in capabilities of HTTPS and SSL.  AMF as a format is meant to ride on top of a RESTful architecture such as HTTP.  By way of this you can hide any of the database services from the outside world and simply filter and process requests once they come in from the AMF services.  Along with this AMF can serve as another layer of authentication between client and server with cookie and ticket verifications above and beyond HTTP authentication.

The concept of where the code lives is pretty important.  In the case without AMF, would you include the database calls and ODBC logic directly in a client distributable application?  Wouldn’t a better idea be to include that logic on the server side and simply have the client call a single method from the server which then does the database calls?  This is very similar to having something like PHP locally make requests to a database, however in the case of AMF it is a rich client (Flash) passing objects back and forth.  Anyways.. enough this conversation can go on forever.  The important thing to know about AMF from my perspective is that it is RESTful so it relies of HTTP/S architecture, and allows the remote execution of methods.

Below are a handful of places to get information and existing assemblies that allow for AMF calls.  I wrote my own implementation because I couldn’t find anything out there that worked correctly from Powershell.  What I have done is pretty focused towards a certain application and it’s limited scope of objects it required to pass with AMF.  Adding new types of objects is a matter of 6-8 lines of code now.  Others out there like FlourineFX seem to be very mature in the breadth of capabilities.

Action Message Format – Wikipedia, the free encyclopedia

AMF 3 – Adobe Open Source

Java AMF Client – BlazeDS – Adobe Open Source

FluorineFx Home