Event: BeerCityCode 2018 Workshop

Join us for BeerCityCode.com  –  June 22-23, 2018  –  Grand Rapids, MichiganBeerCityCode

Robin and Paul Schroeder will be presenting a 6 hour workshop on Rapid Delivery of Enterprise Architectures

Beware ye of dangers that lurk in “Exampleware” software! Understand how patterns and practices form an Enterprise Architecture stack that adheres to S.O.L.I.D. principles – with a twist. Use code generation techniques to do it! It is time to learn a few new tricks and up your game!

Enterprise Architecture

Beware ye of dangers that lurk in “Exampleware” software! In this workshop, the dynamic duo of Paul (aka Batman) and Robin and will walk participants through the creation of a S.O.L.I.D. software enterprise architecture using Microsoft .NET. Code generation will be demonstrated as a technique that enables instantaneous proof-of-concept development and quick iteration.

Attendees will have their choice of pursuing either a web (ASP.NET) or mobile (Xamarin) front-end.

  • Attendees that want more of a hackathon experience are encouraged to think of their own idea to develop.
  • Assistance creating initial relational database tables to use throughout the sessions is provided, as needed.
  • Those more interested in just the material and techniques are provided with an open source sample database schema to use.

Software Patterns:

Each session covers one or more common software patterns. We will explain why they are important, how to separate the concerns into layers, and how they work together.

  1. Database schema and relationships
  2. Repository pattern
  3. Mapper pattern
  4. Interfaces (Dependency injection and Mocks)
  5. Data Transfer Objects
  6. Web API (RESTful implementation)
  7. Service Provider (Data Service)
  8. Domain Models vs. Data Models

Time permitting and, depending on audience interest, one or more of the following may also be covered:

  • MVC pattern (ASP.NET / Core)
  • MVVM pattern (mobile / Xamarin)
  • OWIN pipeline
  • Security (JSON Web Tokens)

Code Generation:

Participants are provided with a free one-year trial license for the code generation tool that will be used in demonstrations, CodeGenHero. This Visual Studio extension is used to:

  1. Reverse-engineer database schema into metadata.
  2. Auto-generate code in each layer of the architecture, as it is covered.

Prerequisites:

  • Visual Studio 2017
  • Intermediate to advanced knowledge of C#
  • Local SQL Server instances (Express, Developer, etc.) are encouraged, but not required. Temporary Azure SQL DBs will be provided, as needed.

The Takeaway

Understand how patterns and practices form a software Enterprise Architecture stack that adheres to S.O.L.I.D. principles. Learn code generation techniques that dramatically improve developer productivity. This workshop helps you up your game, break the Iron Triangle, and become a hero within your own organization.

 

Event: CNUG Rapid Deliver of Enterprise Architecture – Secrets of Code Generation

Chicago .Net User Group (CNUG) Meetup Posting

Learn how to dramatically improve developer productivity at the start of a project using code generation techniques. Understand how patterns can come together to form an Enterprise Architecture stack that adheres to SOLID principles. Overcome the “It’s too abstract (hard)” or “Too much time” arguments by combining distilled patterns with code generation that results in applications that are more “-able” (test-able, adapt-able, maintain-able) throughout the life of any software system. Paul’s CodeGenHero techniques draw upon his decades of experience and go beyond theory. He will demonstrate how universal rules of software architecture can be applied to radically diminish the cost of design changes. See how publicly-available, open source, foundation classes can be leveraged to meet your own challenges of how to accelerate development while simultaneously maintaining code quality.

In this session:

    • Learn what software architects need to achieve for enterprise applications
    • Define appropriate boundaries and layers to organize components and services
    • Discover code generation principles that can make you a better developer

 

 

Speakers: Paul Schroeder and Robin Schroeder

Nik Patel also spoke on the same night about postman. We worked with Nik such that he was able to use our BingoBuzz APIs to demonstrate during his really great talk on Postman.

Develop API Test Suite using Postman
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
In a world of API driven development, API tests are crucial to assert working Apis in integrated environments. This presentation will cover how Postman allows creating API Tests with the least amount of effort. We will go through how to,
 Quickly make API calls and inspect responses
 Use environment and global variables
 Write tests using JavaScript
 Create Postman collections
 Add Postman Test suite to CI/CD pipeline using Newman

By the end of this presentation, you will be ready to start using postman as a power user to create E2E API tests for your Apis.

Speaker : Nikunj Patel

NuGet ContentFiles Demystified

Actually, that title is meant to be, “NuGet ContentFiles Demystified”
beta-ray-bill

If there is nothing but what we make in this world, brothers…let us make good.” -Beta Ray Bill, Marvel Comics


An important step has been made in the ongoing voyage of making CodeGenHero (CGH) easy to implement and flexible to use – use of NuGet packages as a means of deployment!

Install-Package CodeGenHero.WebApi

Along the lines of the above quote, developers should have the ability not only to use CGH infrastructure classes and interfaces, but to improve upon them and customize their implementations.  Thus, I set about what became a rather frustrating experience into the attempt to simply include CGH class files as part of NuGet packages.  In the end, it seems that providing additional interfaces and ensuring items can be inherited for potentially overriding behavior became the preferred approach.

This decision was not without side effects.  This post will cover some of those effects and also provide a workaround for others that may want to include files in their NuGet packages targeting .NET Standard or .NET Core projects.

Villain: No Mutable Class Files in NuGet Packages

All I really want to do is, upon installing a CodeGenHero NuGet package, have that package add some “starter” or base class files to the project that the user can either keep as is, or customize as their project evolves.  There is/was support for this functionality in prior versions, but it appears to have been taken away for the newer ProjectReference NuGet format (.NET Standard class library or .NET Core projects come with PackageReference enabled by default).

Technically, this is not 100% accurate, as I found a hack that allows me to include such files, but it has an unacceptable side-effect; described at the end of this post.  Also, it appears I am trying to do something that is a no-no – include mutable files in a NuGet package. Here are some excerpts and links describing the issue:

https://blog.nuget.org/20160126/nuget-contentFiles-demystified.html

ContentFiles in NuGet are static files that the NuGet client will make available through the project.lock.json file [emphasis added] to a project for inclusion in the project.

This feature is only for packages that will be installed to projects that are managed using a project.json file. Currently only two projects types are managed by a project.json.

  • Portable class libraries
  • UWP apps

The contentFiles option is not available for other project types.”

https://docs.microsoft.com/en-us/nuget/reference/nuspec#including-content-files

Holy Covfefe Batman

“Content files are immutable files that a package needs to include in a project. Being immutable, they are not intended to be modified by the consuming project.”[emphasis added]

Well, “Holy Covfefe Batman”, this puts a wrinkle in the plan!

Antihero: More interfaces, conversion of enums to class files

Some code I wanted to add to target projects were enumerations. Because enumerations cannot inherit from other enumerations, I opted to convert these to classes with virtual method implementations.  The approach seems odd to me, but it is one way to solve the issue.  For example, when logging, users should be able to extend the “LogMessageType” item, which contains a way to record numeric codes in the log for subsequent query filtering.  What was formerly an enumeration became a class file:

public enum LogMessageType

became:

public class LogMessageType

Such items do not need to have an instance for every time they are used and so a singleton pattern has also been applied to the new classes. As an example, the usage now looks like:

LogMessageType.Instance.Warn_WebApiClient

I’m not super happy about it, but it does preserve the ability for the end user to extend the LogMessageType now that it won’t be added as a source file to the consuming project.

In addition, without being able to add mutable class files to NuGet packages, additional interfaces were added so consumers of the packages can completely swap out behavior, if needed.  For example, the generic HttpCallResult<Tclass now implements the IHttpCallResultCGHT interface (adding “CGHT” to eliminate chance conflicts with System.Web.Http).  Likewise, the generic PageData<Tclass now implements the IPageDataT<Tinterface.

With these workarounds in place, CodeGenHero can provide all the goodness of easy-to-install NuGet packages while remaining extensible.

Epilogue: Using NuGet ContentFiles

Following is a quick summation of findings for readers encountering this post that want to leverage NuGet Content Files for static content, or regardless, include cs code files in their NuGet packages.

Three scenarios were observed when installing the CodeGenHero.Repository.EntityFramework NuGet package containing three content files.

  1. If installing into a .NET Standard 2.0 project, the ReversePOCO and DSN folders are created, along with the contained files (DSNContext.tt, EF.Reverse.POCO.Core.ttinclude, and EF.Reverse.POCO.ttinclude).
    • However, the files are created as shortcuts pointing back to the NuGet installation folder.
  2. If installing into a full .NET framework project that uses the packages.config NuGet reference format, the ReversePOCO and DSN folders are created, along with the contained files (DSNContext.tt, EF.Reverse.POCO.Core.ttinclude, and EF.Reverse.POCO.ttinclude).
    • The files are local to the project and this is exactly the original intent.
  3. If installing into a full .NET framework project that uses the PackageReference NuGet reference format (in csproj), the ReversePOCO and DSN folders are not created.
    • The content files contained in the NuGet package are ignored (DSNContext.tt, EF.Reverse.POCO.Core.ttinclude, and EF.Reverse.POCO.ttinclude).

 

Errata:

Below is an “amusing” item, found as I sifted through a plethora of outdated documentation that often was less than a year old.  This post, seemingly created just over a month prior to my current adventure was already deprecated!

01/18/2018 – Impact of project.json when creating packages

https://docs.microsoft.com/en-us/nuget/archive/project-json-impact

This content is deprecated. Projects should use either the packages.config or PackageReference formats.

Changes affecting existing packages usage – Traditional NuGet packages support a set of features that are not carried over to the transitive world. The transitive restore model, described in Dependency resolution, does not have a concept of “package install time”. A package is either present or not present, but there is no consistent process that occurs when a package is installed.

Also, install scripts were supported only in Visual Studio. Other IDEs had to mock the Visual Studio extensibility API to attempt to support such scripts, and no support was available in common editors and command-line tools. 

Here is one of the most straightforward blog posts I found on Multi-Targeting and Porting a .NET Library to .NET Standard.  It is by Rick Strahl and contains a section on how to enable NuGet package creation directly from within Visual Studio projects:

https://weblog.west-wind.com/posts/2017/Jun/22/MultiTargeting-and-Porting-a-NET-Library-to-NET-Core-20

If you are interested in Targeting multiple .NET platforms in a single NuGet package with Visual Studio 2017, Bart Wolff also has a blog post worth mentioning:

https://www.bartwolff.com/Blog/2017/03/20/targeting-multiple-net-platforms-in-a-single-nuget-package-w

Here are excerpts of an explanation why including mutable files in NuGet packages has become a no-no.

March 2017 – https://github.com/NuGet/Home/issues/4803

Question: For our packages it is important that the content files (assets) are copied to the solution, not just referenced. With the package we need to copy config files (XML files) the developer wants to change. This was standard in the packages.config era.

Response: This is not supported with project.json/PackageReference. With packages.config users had to run install and uninstall commands explicitly and this was done through Visual Studio.

With project.json/PackageReference packages can float which allows them to change from restore to restore without an explicit install/uninstall action being performed. For this reason the contentFiles folder is immutable and cannot carry a state like the packages.config content folder.

The plan going forward for this is to allow users to move files from under the contentFiles folder into their project through a gesture in Visual Studio. Currently the include of these files is written out to the auto generated nuget props file in the obj folder which allows them to be removed or overridden.

My suggestion for copying config XML files to the user project is to handle this in an init.ps1 script, or possibly in a .targets file.

Question: init.ps1 and targets work in ASP.NET Core projects (csproj)?

Answer: Yes, if installed using Visual Studio init.ps1 will be executed. This works on all types of projects. The script will also run when opening the solution if the nuget powershell console is open.

So, in the end, it may be possible to do what I was trying to do by writing some  PowerShell scripts, but at that point I was questioning whether it was worth continuing with such an approach.  It seemed like trying to swim upriver and prone to issues as users upgraded versions of individual packages on top of their customized CodeGenHero class files.

Here are some interesting items to note as well.  In the PackageReference format, it is possible to include files in the NuGet package – and even get them copied into the  project.  To do this, add sections that look like this to your csproj file:

<ItemGroup>
 <Content Include="contentFiles/**/*.*">
 <IncludeInPackage>true</IncludeInPackage>
 <CopyToOutput>false</CopyToOutput>
 <BuildAction>Compile</BuildAction>
 <CopyToOutputDirectory>Always</CopyToOutputDirectory>
 </Content>
</ItemGroup>

   

What I did not like about this is, although NuGet would add the files to my project, it seemed to include them as shortcuts with “update” links pointing back to the NuGet installation folder:

  <ItemGroup>

    <Compile Update=C:\Users\paul\.nuget\packages\codegenhero.repository.automapper\1.0.0\contentFiles\any\netstandard2.0\Infrastructure\AutoMapperInitializer.cs>

      <CopyToOutputDirectory>Never</CopyToOutputDirectory>

    </Compile>

    <Compile Update=C:\Users\paul\.nuget\packages\codegenhero.repository.automapper\1.0.0\contentFiles\any\netstandard2.0\Infrastructure\GenericFactory.cs>

      <CopyToOutputDirectory>Never</CopyToOutputDirectory>

    </Compile>

    <Compile Update=C:\Users\paul\.nuget\packages\codegenhero.repository.automapper\1.0.0\contentFiles\any\netstandard2.0\Interfaces\IGenericFactory.cs>

      <CopyToOutputDirectory>Never</CopyToOutputDirectory>

    </Compile>

  </ItemGroup>

When I tried a sample found online, it exhibited the same shortcut behavior, but without the update links.  Despite technically getting the files to copy into the project, this was not going to be a solution I could be comfortable using.

https://www.nuget.org/packages/ContentFilesExample/

ContentFilesExample

Running the “Save As” command on
the “ExampleInternals.cs” file shows it is pointing to my packages directory:

C:\Users\paul\.nuget\packages\contentfilesexample\1.0.2\contentFiles\cs\netstandard1.0\ExampleInternals.cs

Other related links:

contentFiles (cs, compile) not working in NetStandard projects.

https://github.com/NuGet/Home/issues/4803

Packing static content in Nuget for PackageReferece projects

https://stackoverflow.com/questions/48751019/packing-static-content-in-nuget-for-packagereferece-projects

How to restore cshtml from nuget packages. #1490

https://github.com/aspnet/Home/issues/1490

NuGet is now fully integrated into MSBuild

https://blog.nuget.org/20170316/NuGet-now-fully-integrated-into-MSBuild.html

Create and publish a package using Visual Studio

https://docs.microsoft.com/en-us/nuget/quickstart/create-and-publish-a-package-using-visual-studio

Supporting multiple .NET framework versions

https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks

How to specify target frameworks

https://docs.microsoft.com/en-us/dotnet/standard/frameworks#how-to-specify-target-frameworks

Creating NuGet packages

https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package#from-a-convention-based-working-directory

.nuspec reference

https://docs.microsoft.com/en-us/nuget/reference/nuspec#including-content-files

Adding nuget pack as a msbuild target

https://github.com/NuGet/Home/wiki/Adding-nuget-pack-as-a-msbuild-target

 Package references (PackageReference) in project files

https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files

Entity Framework Combines Forces with AutoMapper to Unleash Evil StackOverflowException

Thor“Our ancestors called it magic but you call it science. I come from a land where they are one and the same.”

– Chris Hemsworth, Thor

To non-developers, computer programming must sometimes seem akin to magic.  Heck, even after years of being a .NET developer, I still marvel at the symphonies of code we create.  I love to see how the numerous NuGet/Bower packages of various versions weave together with our own code as glue to orchestrate amazing, impactful, applications.  At least when things work – and when they don’t those same components can be a nightmare!  Such was the case during a recent all-nighter that started with a StackOverflowException.

Spoiler alert – Entity Framework (EF) “Relationship Fixup” feature turned out to be the root of the problem.  I was experimenting with CodeGenHero’s data transfer object (DTO) template to have it include navigation properties – forward and reverse navigation properties.  It did such a good job generating code with parent objects pointing to child objects and child objects having references back to their parents.  Cool, I thought, now I can design API calls with the exact object graph of data I want in a single round trip.  Unfortunately, AutoMapper had other plans – moments after the Web API went to serialize the EF object returned by the repository classes into my DTOs, the server would throw a StackOverflowException.  What the heck?

Villain: Circular Dependency – as a simple example consider the following class:

namespace CircularDependency
{
  public class Person
  {
    public Person(string firstName, string lastName, Person parent)
    {
      FirstName = firstName;
      LastName = lastName;
      Children = new List<Person>();
      Parent = parent;
    }

    public string FirstName { get; set; }
    public string LastName { get; set; }
    public List<Person> Children { get; set; }
    public Person Parent { get; set; }
  }

}

The example above depicts a ‘Person’ class with references to both a parent as well as a list of children.  Obviously, an instance of a parent person object may have multiple references to person objects that contain references back to the same parent, creating a circular dependency.  Other scenarios could involve different classes altogether.  For example, an ‘Order’ could reference a ‘Line Item’, which has a reference back to the order.  Unfortunately, this set-up will cause the wizardry of AutoMapper to conflict with the magic of Entity Framework.

The problem is that AutoMapper could not tell when to stop serializing the data.  This is not a new issue and there are stated solutions out there such as using the MaxDepth() and/or the PreserveReferences features.  However, for whatever reason, this approach was not working and, besides, I didn’t want to limit the depth.  What I wanted was the exact data I told EF to load and nothing more.  Server side, I had invested time in craftily assembling multiple bits of data into a single object graph that I wanted to send across the wire to my client in a single call (the traditional use for DTOs).

I started searching for a way to turn off or disable EF’s “relationship fixup” feature and found that it could not be disabled.  Call me stubborn or consider me persistent, either way I wasn’t going to let EF dictate my design!  So, I hunkered down and began “Operation Relationship Breakup” – my attempt to undo EF’s relationship fix-up feature (clever, I know).  The aim was to detect the first circular reference of each property and IEnumerable<T> and then null out its reference.  After several hours and multiple reflection algorithms, I had to throw in the towel.  Even when my algorithm came close to the desired behavior of nulling out EF’s circular references the performance impact was unacceptable.  I was adding overhead to undo something EF already burned cycles establishing.

I rationalized that many developers consider object designs with circular references to be bad practice, anyhow.  Thus, I set about removing the bi-directional reference properties in the DTO’s to only go one‑way in the object graph.  What this means, in the example below, is that the Person object might have a list of children and a primary key value for the parent, but it will not have a reference-based parent property.

Antihero: One-Way DTOs – no reference-based parent property:

namespace CircularDependency
{
  public class Person
  {
    public Person(string firstName, string lastName, int parentId)
    {
      FirstName = firstName;
      LastName = lastName;
      Children = new List<Person>();
      ParentId = parentId;
    }

    public string FirstName { get; set; }
    public string LastName { get; set; }
    public List<Person> Children { get; set; }
    public int ParentId { get; set; }
  }
}

 

In the end, once the DTOs have one-way navigation then AutoMapper and Entity Framework play nice once again and I can go back to being productive.

Here’s a couple of relevant links:

https://stackoverflow.com/questions/37251043/automapper-throwing-stackoverflowexception-when-calling-projecttot-on-iquery

https://stackoverflow.com/questions/31107876/how-should-i-disable-entity-framework-table-refrenceforeign-list-from-each-obj