General .Net – CodeGenHero

Actually, that title is meant to be, “NuGet ContentFiles Demystified”
beta-ray-bill

“If there is nothing but what we make in this world, brothers…let us make good.” -Beta Ray Bill, Marvel Comics

An important step has been made in the ongoing voyage of making CodeGenHero (CGH) easy to implement and flexible to use – use of NuGet packages as a means of deployment!

Install-Package CodeGenHero.WebApi

Along the lines of the above quote, developers should have the ability not only to use CGH infrastructure classes and interfaces, but to improve upon them and customize their implementations. Thus, I set about what became a rather frustrating experience into the attempt to simply include CGH class files as part of NuGet packages. In the end, it seems that providing additional interfaces and ensuring items can be inherited for potentially overriding behavior became the preferred approach.

This decision was not without side effects. This post will cover some of those effects and also provide a workaround for others that may want to include files in their NuGet packages targeting .NET Standard or .NET Core projects.

Villain: No Mutable Class Files in NuGet Packages

All I really want to do is, upon installing a CodeGenHero NuGet package, have that package add some “starter” or base class files to the project that the user can either keep as is, or customize as their project evolves. There is/was support for this functionality in prior versions, but it appears to have been taken away for the newer ProjectReference NuGet format (.NET Standard class library or .NET Core projects come with PackageReference enabled by default).

Technically, this is not 100% accurate, as I found a hack that allows me to include such files, but it has an unacceptable side-effect; described at the end of this post. Also, it appears I am trying to do something that is a no-no – include mutable files in a NuGet package. Here are some excerpts and links describing the issue:

https://blog.nuget.org/20160126/nuget-contentFiles-demystified.html

“ContentFiles in NuGet are static files that the NuGet client will make available through the project.lock.json file [emphasis added] to a project for inclusion in the project.

This feature is only for packages that will be installed to projects that are managed using a project.json file. Currently only two projects types are managed by a project.json.

Portable class libraries
UWP apps

The contentFiles option is not available for other project types.”

https://docs.microsoft.com/en-us/nuget/reference/nuspec#including-content-files

Holy Covfefe Batman

“Content files are immutable files that a package needs to include in a project. Being immutable, they are not intended to be modified by the consuming project.”[emphasis added]

Well, “Holy Covfefe Batman”, this puts a wrinkle in the plan!

Antihero: More interfaces, conversion of enums to class files

Some code I wanted to add to target projects were enumerations. Because enumerations cannot inherit from other enumerations, I opted to convert these to classes with virtual method implementations. The approach seems odd to me, but it is one way to solve the issue. For example, when logging, users should be able to extend the “LogMessageType” item, which contains a way to record numeric codes in the log for subsequent query filtering. What was formerly an enumeration became a class file:

public enum LogMessageType

became:

public class LogMessageType

Such items do not need to have an instance for every time they are used and so a singleton pattern has also been applied to the new classes. As an example, the usage now looks like:

LogMessageType.Instance.Warn_WebApiClient

I’m not super happy about it, but it does preserve the ability for the end user to extend the LogMessageType now that it won’t be added as a source file to the consuming project.

In addition, without being able to add mutable class files to NuGet packages, additional interfaces were added so consumers of the packages can completely swap out behavior, if needed. For example, the generic HttpCallResult<T> class now implements the IHttpCallResultCGHT interface (adding “CGHT” to eliminate chance conflicts with System.Web.Http). Likewise, the generic PageData<T> class now implements the IPageDataT<T> interface.

With these workarounds in place, CodeGenHero can provide all the goodness of easy-to-install NuGet packages while remaining extensible.

Epilogue: Using NuGet ContentFiles

Following is a quick summation of findings for readers encountering this post that want to leverage NuGet Content Files for static content, or regardless, include cs code files in their NuGet packages.

Three scenarios were observed when installing the CodeGenHero.Repository.EntityFramework NuGet package containing three content files.

If installing into a .NET Standard 2.0 project, the ReversePOCO and DSN folders are created, along with the contained files (DSNContext.tt, EF.Reverse.POCO.Core.ttinclude, and EF.Reverse.POCO.ttinclude).
- However, the files are created as shortcuts pointing back to the NuGet installation folder.
If installing into a full .NET framework project that uses the packages.config NuGet reference format, the ReversePOCO and DSN folders are created, along with the contained files (DSNContext.tt, EF.Reverse.POCO.Core.ttinclude, and EF.Reverse.POCO.ttinclude).
- The files are local to the project and this is exactly the original intent.
If installing into a full .NET framework project that uses the PackageReference NuGet reference format (in csproj), the ReversePOCO and DSN folders are not created.
- The content files contained in the NuGet package are ignored (DSNContext.tt, EF.Reverse.POCO.Core.ttinclude, and EF.Reverse.POCO.ttinclude).

Errata:

Below is an “amusing” item, found as I sifted through a plethora of outdated documentation that often was less than a year old. This post, seemingly created just over a month prior to my current adventure was already deprecated!

01/18/2018 – Impact of project.json when creating packages

https://docs.microsoft.com/en-us/nuget/archive/project-json-impact

“This content is deprecated. Projects should use either the packages.config or PackageReference formats.”

Changes affecting existing packages usage – Traditional NuGet packages support a set of features that are not carried over to the transitive world. The transitive restore model, described in Dependency resolution, does not have a concept of “package install time”. A package is either present or not present, but there is no consistent process that occurs when a package is installed.

Also, install scripts were supported only in Visual Studio. Other IDEs had to mock the Visual Studio extensibility API to attempt to support such scripts, and no support was available in common editors and command-line tools.

Here is one of the most straightforward blog posts I found on Multi-Targeting and Porting a .NET Library to .NET Standard. It is by Rick Strahl and contains a section on how to enable NuGet package creation directly from within Visual Studio projects:

https://weblog.west-wind.com/posts/2017/Jun/22/MultiTargeting-and-Porting-a-NET-Library-to-NET-Core-20

If you are interested in Targeting multiple .NET platforms in a single NuGet package with Visual Studio 2017, Bart Wolff also has a blog post worth mentioning:

https://www.bartwolff.com/Blog/2017/03/20/targeting-multiple-net-platforms-in-a-single-nuget-package-w

Here are excerpts of an explanation why including mutable files in NuGet packages has become a no-no.

March 2017 – https://github.com/NuGet/Home/issues/4803

Question: For our packages it is important that the content files (assets) are copied to the solution, not just referenced. With the package we need to copy config files (XML files) the developer wants to change. This was standard in the packages.config era.

Response: This is not supported with project.json/PackageReference. With packages.config users had to run install and uninstall commands explicitly and this was done through Visual Studio.

With project.json/PackageReference packages can float which allows them to change from restore to restore without an explicit install/uninstall action being performed. For this reason the contentFiles folder is immutable and cannot carry a state like the packages.config content folder.

The plan going forward for this is to allow users to move files from under the contentFiles folder into their project through a gesture in Visual Studio. Currently the include of these files is written out to the auto generated nuget props file in the obj folder which allows them to be removed or overridden.

My suggestion for copying config XML files to the user project is to handle this in an init.ps1 script, or possibly in a .targets file.

Question: init.ps1 and targets work in ASP.NET Core projects (csproj)?

Answer: Yes, if installed using Visual Studio init.ps1 will be executed. This works on all types of projects. The script will also run when opening the solution if the nuget powershell console is open.

So, in the end, it may be possible to do what I was trying to do by writing some PowerShell scripts, but at that point I was questioning whether it was worth continuing with such an approach. It seemed like trying to swim upriver and prone to issues as users upgraded versions of individual packages on top of their customized CodeGenHero class files.

Here are some interesting items to note as well. In the PackageReference format, it is possible to include files in the NuGet package – and even get them copied into the project. To do this, add sections that look like this to your csproj file:

<ItemGroup>
 <Content Include="contentFiles/**/*.*">
 <IncludeInPackage>true</IncludeInPackage>
 <CopyToOutput>false</CopyToOutput>
 <BuildAction>Compile</BuildAction>
 <CopyToOutputDirectory>Always</CopyToOutputDirectory>
 </Content>
</ItemGroup>

What I did not like about this is, although NuGet would add the files to my project, it seemed to include them as shortcuts with “update” links pointing back to the NuGet installation folder:

<CopyToOutputDirectory>Never</CopyToOutputDirectory>

</Compile>

<CopyToOutputDirectory>Never</CopyToOutputDirectory>

</Compile>

<CopyToOutputDirectory>Never</CopyToOutputDirectory>

</Compile>

</ItemGroup>

When I tried a sample found online, it exhibited the same shortcut behavior, but without the update links. Despite technically getting the files to copy into the project, this was not going to be a solution I could be comfortable using.

https://www.nuget.org/packages/ContentFilesExample/

ContentFilesExample

Running the “Save As” command on
the “ExampleInternals.cs” file shows it is pointing to my packages directory:

C:\Users\paul\.nuget\packages\contentfilesexample\1.0.2\contentFiles\cs\netstandard1.0\ExampleInternals.cs

Other related links:

contentFiles (cs, compile) not working in NetStandard projects.

https://github.com/NuGet/Home/issues/4803

Packing static content in Nuget for PackageReferece projects

https://stackoverflow.com/questions/48751019/packing-static-content-in-nuget-for-packagereferece-projects

How to restore cshtml from nuget packages. #1490

https://github.com/aspnet/Home/issues/1490

NuGet is now fully integrated into MSBuild

https://blog.nuget.org/20170316/NuGet-now-fully-integrated-into-MSBuild.html

Create and publish a package using Visual Studio

https://docs.microsoft.com/en-us/nuget/quickstart/create-and-publish-a-package-using-visual-studio

Supporting multiple .NET framework versions

https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks

How to specify target frameworks

https://docs.microsoft.com/en-us/dotnet/standard/frameworks#how-to-specify-target-frameworks

Creating NuGet packages

https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package#from-a-convention-based-working-directory

.nuspec reference

https://docs.microsoft.com/en-us/nuget/reference/nuspec#including-content-files

Adding nuget pack as a msbuild target

https://github.com/NuGet/Home/wiki/Adding-nuget-pack-as-a-msbuild-target

Package references (PackageReference) in project files

https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files

Thor “Our ancestors called it magic but you call it science. I come from a land where they are one and the same.”

– Chris Hemsworth, Thor

To non-developers, computer programming must sometimes seem akin to magic. Heck, even after years of being a .NET developer, I still marvel at the symphonies of code we create. I love to see how the numerous NuGet/Bower packages of various versions weave together with our own code as glue to orchestrate amazing, impactful, applications. At least when things work – and when they don’t those same components can be a nightmare! Such was the case during a recent all-nighter that started with a StackOverflowException.

Spoiler alert – Entity Framework (EF) “Relationship Fixup” feature turned out to be the root of the problem. I was experimenting with CodeGenHero’s data transfer object (DTO) template to have it include navigation properties – forward and reverse navigation properties. It did such a good job generating code with parent objects pointing to child objects and child objects having references back to their parents. Cool, I thought, now I can design API calls with the exact object graph of data I want in a single round trip. Unfortunately, AutoMapper had other plans – moments after the Web API went to serialize the EF object returned by the repository classes into my DTOs, the server would throw a StackOverflowException. What the heck?

Villain: Circular Dependency – as a simple example consider the following class:

namespace CircularDependency
{
  public class Person
  {
    public Person(string firstName, string lastName, Person parent)
    {
      FirstName = firstName;
      LastName = lastName;
      Children = new List<Person>();
      Parent = parent;
    }

    public string FirstName { get; set; }
    public string LastName { get; set; }
    public List<Person> Children { get; set; }
    public Person Parent { get; set; }
  }

}

The example above depicts a ‘Person’ class with references to both a parent as well as a list of children. Obviously, an instance of a parent person object may have multiple references to person objects that contain references back to the same parent, creating a circular dependency. Other scenarios could involve different classes altogether. For example, an ‘Order’ could reference a ‘Line Item’, which has a reference back to the order. Unfortunately, this set-up will cause the wizardry of AutoMapper to conflict with the magic of Entity Framework.

The problem is that AutoMapper could not tell when to stop serializing the data. This is not a new issue and there are stated solutions out there such as using the MaxDepth() and/or the PreserveReferences features. However, for whatever reason, this approach was not working and, besides, I didn’t want to limit the depth. What I wanted was the exact data I told EF to load and nothing more. Server side, I had invested time in craftily assembling multiple bits of data into a single object graph that I wanted to send across the wire to my client in a single call (the traditional use for DTOs).

I started searching for a way to turn off or disable EF’s “relationship fixup” feature and found that it could not be disabled. Call me stubborn or consider me persistent, either way I wasn’t going to let EF dictate my design! So, I hunkered down and began “Operation Relationship Breakup” – my attempt to undo EF’s relationship fix-up feature (clever, I know). The aim was to detect the first circular reference of each property and IEnumerable<T> and then null out its reference. After several hours and multiple reflection algorithms, I had to throw in the towel. Even when my algorithm came close to the desired behavior of nulling out EF’s circular references the performance impact was unacceptable. I was adding overhead to undo something EF already burned cycles establishing.

I rationalized that many developers consider object designs with circular references to be bad practice, anyhow. Thus, I set about removing the bi-directional reference properties in the DTO’s to only go one‑way in the object graph. What this means, in the example below, is that the Person object might have a list of children and a primary key value for the parent, but it will not have a reference-based parent property.

Antihero: One-Way DTOs – no reference-based parent property:

namespace CircularDependency
{
  public class Person
  {
    public Person(string firstName, string lastName, int parentId)
    {
      FirstName = firstName;
      LastName = lastName;
      Children = new List<Person>();
      ParentId = parentId;
    }

    public string FirstName { get; set; }
    public string LastName { get; set; }
    public List<Person> Children { get; set; }
    public int ParentId { get; set; }
  }
}

In the end, once the DTOs have one-way navigation then AutoMapper and Entity Framework play nice once again and I can go back to being productive.

Here’s a couple of relevant links:

https://stackoverflow.com/questions/37251043/automapper-throwing-stackoverflowexception-when-calling-projecttot-on-iquery

https://stackoverflow.com/questions/31107876/how-should-i-disable-entity-framework-table-refrenceforeign-list-from-each-obj

CodeGenHero

Category: General .Net

NuGet ContentFiles Demystified

Entity Framework Combines Forces with AutoMapper to Unleash Evil StackOverflowException