I recently posted a comment about the evolution of Example42 modules in the Example42 Puppet Modules Google group.

 

It contains my opinions on how modules should be and how could evolve the new generation of Puppet modules.

 

I repost it here, as it gives a good overview of my current ideas and the evolution of Example42 modules.

 

--- 

 

Let's move.

This is not something that I can do alone.

NextGen modules are there, somehow stable, doing their work.

They are requiring a lot of support that I can't give in my free time.

Even for testing submitted patches.

 

We have to start to work on brand new 3.x modules and a sane ecosystem.

 

I suggest these principles:

0- No NextGen legacies 

1- Real single point of responsability for modules

2- Higher abstraction  stack modules

3- Follow stdmod naming conventions

4- Distributed development (modules maintainers)

5- Implementation and usage patterns coherency

 

 

0- No NextGen legacies

 

The current NextGen modules should not be taken as reference, no needs to have backwards compatibility.

Let's take the good things (defines, implementation patterns, correct params and setups for different OS ) and build from them.

Let's cut all the parameters and functionalities that don't fit with points 1 and 3.

 

 

1- Real single point of responsability for modules

 

This is important and critical for the whole long term feasibility and reusability of the modules:

an application module should only expose what's needed to configure and manage its application.

No relation to third modules, no options to provide implementation logics of higher abstractions.

For example, an apache main class should expose just the essential parameters to just manage (in a reusable way) the application.

Then the module may have custom defines to manage modules, virtualhosts, users, or whatever.

We can reuse the good ones around, adapting the parameters to stdmod conventions.

It might even have something like apache::passenger, with its own parameters, but is should not expose any passenger related parameter in the main class.

I would add to these basic parameters the ones need to firewall and monitor them, keeping them in the most simple and open way, just 2 parameters:

$monitor_class - The name of the class to use to manage monitoring.

$monitor_config_hash - An open to any option configuration hash (we may stanrdardize it when working on the new monitor / firewall implementations ) 

Same for $firewall_

I'd also provide in each module (that needs them) standard "general purpose" defines to:

- Manage single configuration files (useful when multiple config files are needed and you don't want to clutter the main class params to manage them)

- Manage single configuration entries (via augeas?)

No read need, IMHO, to have different defines with specific parameters for different configuration files, just assure there's a "catchall" config_hash parameter.

So just dumb interfaces to the application's configuration , all the logic and the implementation details can be provided by 2:

This would allow quick creation of basic modules with at least these "minimal" features, then we can add or import more specific ones. 

Here we have to place rspec tests, rspec-system tests and essential documentation (all the stdmod standards related one can be avoided, imho)

 

2- Higher abstraction  stack modules

 

Here we move all the logic to manage modules interdipendences and integrations.

So we might have modules like:

puppet-stack_lamp

puppet-stack_liferay (but also just puppet-liferay as long as it follows stacks' patterns)

puppet-stack_rails

puppet-stack_logs ( here's a sample https://github.com/example42/puppet-stack/blob/master/manifests/logs.pp )

puppet-stack_puppetinfrastructure ( https://github.com/example42/puppet-stack_puppet  ) 

Stacks do, in some way, what we often do in our puppet setups (and have called them roles, hostgroups, profiles) exposing higher level parameters that allow and manage the logic provided by the stack to integrate different application modules.

They are supposed to be used by nodes doing different things which are needed to build the stack.

In stacks some parameters could be managed in automatic ways, based on the nodes who use their single components.

With puppetdbquery some wonderful things could be done.

The parameters exposed by a stack class should be about:

- if to include a specific application in the node ( install_webserver ?)

- what to use for each stack's application (where alternatives are provided) ( webserver_name ? ),

- how to configure them (parameters hash, custom template files )

- endpoints, credentials, urls…

Stacks are the place for integration tests. 

 

 

3- Follow stdmod naming conventions

 

They are still not definitive, I really urge whoever is interested in a good shared modules ecosystem to contribute actively to https://github.com/stdmod

We definitively need to decide on few remaining things, IMHO, like:

file (file_path? config_file? config?) (1)

file_source (source? config_file_source? config_source?)

file_template (template? config_file_template? config_template?...)

Since we are doing this as users of Ex42 modules we might decide to choose the current names for the most used parameters (template? source?) as long as they make sense in the whole stdmod picture.

 

 

4- Distributed development (modules maintainers)

 

This can't be an Example42's only work. I can't and don't want to manage multiple dozens of modules of applications I don't know or use.

Once good blueprints are available for different kind of modules, we can quickly create new modules based on them, adding and adapting the available defines, facts, functions from nextgen and  other open source modules.

Possibly involving the relevant authours, also as maintainers.

Code can stay in each maintainer's repos , Dolf, I, whoever can manage their own collections of stdmodules and stacks, pointing to gitsubmodules at the desired version / head (or managing them in a single big git repo)

Of course just one "stdmod" module for application would be the ideal: forks just for pulling back to upstream, as managed by the official maintainer(s).

All this, strictly following point 5:

 

 

5- Implementation and usage patterns coherency

 

It's very important to keep coherency inside the modules set, both for naming and usage patterns.

For this the sooner the finalize naming standars and we have good samples and  templates, the better.

I would stick the code of the modules for Puppet 3 compatibility, no need to do now things that support older versions and too early to embrace fully Puppet 4 ( > 3.3) only  features (data in modules, future parser, containers… ).

They will be useful probably to manage in a smart way firewalling and monitoring, but given the simple basic layout.

For this reason I would leave open parameters like monitor_class or monitor_config_hash that may adapt to any future evolution.

 

 

Next steps…

- propose blueprints for common  module's layout.

- make quick and easy common usage, single package modules (to just "include"): zip, wget, git…

- make essential modules (to improve with time) for the most cross used apps: apache, nginx, mysql…

- explore stacks

my2c

Al