Project.json all the things
One of the less known features of Visual Studio 2015 is that it is possible to use project.json
with any project type, not just “modern PCL’s,” UWP projects, or xproj projects. Read on to learn why you want to switch and how you can update your existing solution.
Background
Since the beginning of NuGet, installed packages were tracked in a file named packages.config
placed alongside the project file. The package installation process goes something like this:
- Determine the full list of packages to install, walking the tree of all dependent packages
- Download all of those packages to a
\packages
directory alongside your solution file - Update your project file with correct libraries from the package (looking at
\lib\TFM
- If the package contains a
build
directory, add any appropriateprops
ortargets
files found
- If the package contains a
- Create or update a
packages.config
file along the project that lists each package along with the current target framework
Terms
- TFM – Target Framework Moniker. The name that represents a specific Platform (platforms being .NET Framework 4.6, MonoTouch, UWP, etc.)
- Short Moniker – a short way of referring to a TFM in a NuGet file (e.g., net46). Full list is here.
- Full Moniker – a longer way of specifying the TFM (e.g., .NETPortable,Version=v4.5,Profile=Profile111). Easiest way to determine this is to compile and let the NuGet error message tell you what to add (see below).
Limitations
The above steps are roughly the same for NuGet up to and including the 2.x series. While it works for basic projects, larger, more complex projects quickly ran into issues. I do not consider the raw number of packages that a project has to be an issue by itself – that is merely showing oodles of reuse and componentization of packages into small functional units. What does become an issue are the UI and the time it takes to update everything.
As mentioned, because NuGet modifies the project file with the relative location of the references, every time you update, it has to edit the project file. This is slow and can lead to merge conflicts across branches.
Furthermore, the system is unable to pivot on different compile-time needs. With many projects needing to provide some native support, NuGet v2.0 had no way of providing different dependencies based on build configuration.
One more issue surfaces with the use of “bait and switch” PCLs. Some packages provide a PCL for reference purpose (the bait), and then also provide platform-specific implementations that have the same external surface area (the switch). This enables libraries to take advantage of platform specific functionality that’s not available in a portable class library alone. The catch with these packages is that to function correctly in a multi-project solution containing a PCL and an application, the application must also add a NuGet reference to all of the packages its PCL libraries use to ensure that the platform-specific version winds up in the output directory. If you forget, you’ll likely get a runtime error due to an incomplete reference assembly being used.
NuGet v3 and Project.json to the rescue
NuGet 3.x introduces a number of new features aimed at addressing the above limitations:
- Project files are no longer modified to contain the library location. Instead, an MSBuild task and target gets auto-included by the build system. This task creates references and content-file items at build time enabling the meta-data values to be calculated and not baked into a project file.
- Per-platform files can exist by using the
runtimes
directories. See the native light-up section in the docs for the details.
- Per-platform files can exist by using the
- Packages are now stored in a per-user cache instead of alongside the solution. This means that common packages do not have to be re-downloaded since they’ll already be present on your machine. Very handy for those packages you use in many different solutions. The MSBuild task enables this as the location is no longer baked into the project file.
- Reference assemblies are now more formalized with a new
ref
top-level directory. This would be the “bait” assembly, one that could target a wide range of frameworks via either aportable-
ordotnet
ornetstandard
TFM. The implementation library would then reside in\lib\TFM
. The version in theref
directory would be used as the compile-time reference while the version in thelib
directory is placed in the output location. - Transitive references. This is a biggie. Now only the top-level packages you require are listed. The full chain of packages is still downloaded (to the shared per-user cache), but it’s hidden in the tooling and doesn’t get in your way. You can continue to focus on the packages you care about. This also works with project-to-project references. If I have a bait-and-switch package reference in my portable project, and I have an application that references that portable library, the full package list will be evaluated for output in the application and the per-architecture, per-platform assemblies will get put in the output directories. You no longer have to reference each package again in the application.
It is important to note that these features only work when a project is using the new project.json
format of package management. Having NuGet v3 alone isn’t enough. The good news is that we can use project.json
in any project type with a few manual steps.
Using project.json in your current solution
You can use project.json
in your current solution. There are a couple of small caveats here:
- Only Visual Studio 2015 with Update 1 currently supports
project.json
. Xamarin Studio does not yet support it but it is planned. That said, Xamarin projects in Visual Studio do supportproject.json
.- If you’re using TFS Team Build, you need TFS 2015 Update 1 on the build agent in addition to VS 2015 Update 1.
- Some packages that rely on
content
files being placed into the project may not work correctly.project.json
has a different mechanism for this, so the package would need to be updated. The workaround would be to manually copy the content into your project file. - All projects in your solution would need to be updated for the transitive references to resolve correctly. That’s to say that an application using NuGet v2/
packages.config
won’t pull in the correct transitive references of a portable project reference that’s usingproject.json
.
With that out of the way, lets get started. If you’d like to skip this and see some examples, please look at the following projects that have been converted over. These are all libraries that have a combination of reference assemblies, platform specific implementations, test applications and unit tests, so the spectrum of scenarios should be covered there. They have everything you need in them:
One last note before diving deep: make sure your .gitignore
file contains the following entries:
*.lock.json
*.nuget.props
*.nuget.targets
These files should not generally be checked in. In particular, the .nuget.props/targets files will contain a per-user path to the NuGet cache. These files are created by calling NuGet restore on your solution file.
Diving deep
As you start, have the following blank project.json
handy as you’ll need it later:
This represents an empty project.json
for a project targeting .NET 4.5.2. I’m using the short moniker here, but you can also use the full one. The string to use here is the thing you’ll likely hit the most trouble with. Fortunately, when you’re wrong and try to build, you’ll get what’s probably the most helpful error message of all time:
Your project is not referencing the “.NETPortable,Version=v4.5,Profile=Profile111” framework. Add a reference to “.NETPortable,Version=v4.5,Profile=Profile111” in the “frameworks” section of your project.json, and then re-run NuGet restore.
The error literally tells you how to fix it. Awesome! The fix is to put .NETPortable,Version=v4.5,Profile=Profile111
in your frameworks section to wind up with something like:
{
"dependencies": {
},
"frameworks": {
".NETPortable,Version=v4.5,Profile=Profile111": { }
},
"supports": { }
}
The eagle-eyed reader will notice that the first example had a runtimes
section with win
in it. This is required for a desktop .NET Framework projects and for projects where CopyNuGetImplementations
is set to true
like your application (we’ll come back that in a bit), but is not required for other library project types. If you have the runtimes
section, then there’s rarely, if ever, a reason to have both the supports
section too.
The easiest way to think about this:
- For library projects, use
supports
and notruntimes
- For your application project, (.exe, .apk, .appx, .ipa, website) use
runtimes
and notsupports
- If it’s a desktop .NET Framework project, use
runtimes
for both class libraries and your application - If it’s a unit test library executing in-place and you need references copied to its output directory, use
runtimes
and notsupports
Now, take note of any packages with the versions that you already have installed. You might want to copy/paste your packages.config
file into a temporary editor window.
The next step is to remove all of your existing packages from your project. There are two ways to do this: via the NuGet package manager console or by hand.
Using the NuGet Package Manager Console
Pull up the NuGet Package Manager Console and ensure the drop-down is set to the project you’re working on. For each package in the project, uninstall each package with the following command:
Uninstall-Package <package name> -Force -RemoveDependencies
Repeat this for each package until they’re all gone.
By Hand
Delete your packages.config
file, save the project file then right-click the project and choose “Unload project”. Now right-click the project and select Edit. We need to clean up a few things in the project file.
- At the top of the project file, remove any
.props
files that were added by NuGet (look for the ones going to a\packages
directory. - Find any
<Reference>
element where theHintPath
points to a NuGet package library. Remove all of them. - At the bottom of the file, remove any
.targets
files that NuGet added. Also remove any NuGet targets or Tasks that NuGet added (might be a target that starts with the following line<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
). - If you have any packages that contain Roslyn Analyzers, make sure to remove any analyzer items that come from them.
Save your changes, right click the project in the solution explorer and reload the project.
Adding the project.json
In your project, add a new blank project.json
file using one of the templates above. Ensure that the Build Action is set to None
(should be the default). Once present, you might need to unload your project and reload it for NuGet to recognize it, so save your project, right-click your project and unload it and reload it.
Now you can either use the Manage NuGet Packages UI to re-add your packages or add them to the project.json
by hand. Remember, you don’t necessarily have to re-add every package, only the top-level ones. For example, if you use Reactive Extensions, you only need Rx-Main
, not the four other packages that it pulls in.
Build your project. If there are any errors related to NuGet, the error messages should guide you to the answer. Your project should build.
What you’ll notice for projects other than desktop .NET executables or UWP appx’s, is that the output directory will no longer contain every referenced library. This saves disk space and helps the build be faster by eliminating extra file copying. If you want the files to be in the output directory, like for unit test libraries that need to execute in-place, or for an application itself, there’s two extra steps to take:
- Unload the project once more and edit it to add the following to the first
<PropertyGroup>
at the top of the project file:<CopyNuGetImplementations>true</CopyNuGetImplementations>
. This tells NuGet to copy all required implementation files to the output directory. - Save and reload the project file. You’ll next need to add that
runtimes
section from above. The exact contents will depend on your project type. Rather than list them all out here, please see the Zeroconf or xUnit for Devices for the full examples.- For an AnyCPU Desktop .NET project
win
is sufficient - For Windows Store projects, you’ll need more
- For an AnyCPU Desktop .NET project
Once you repeat this for all of your projects, you’ll hopefully still have a working build(!) but now one where the projects are using the rich NuGet v3 capabilities. If you have a CI build system, you need to ensure that you’re using the latest nuget.exe
to call restore
on your solution prior to build. My preference is to always download the latest stable version from the dist link here: https://dist.nuget.org/win-x86-commandline/latest/nuget.exe
.
Edge Cases
There may be some edge cases you hit when it comes to the transitive references. If you need to prevent any of the automatic project-to-project propagation of dependencies, the NuGet Docs can help.
In some rare cases, if you start getting compile errors due to missing System
references, you may be hitting this bug, currently scheduled to be fixed in the upcoming 3.4 release. This happens if a NuGet package contains a <frameworkAssembly />
dependency that contains a System.*
assembly. The workaround for now is to add <IncludeFrameworkReferencesFromNuGet>false</IncludeFrameworkReferencesFromNuGet>
to your project file.
What this doesn’t do
There is often confusion between the use of project.json
and its relation to the DNX/CLI project tooling that enables cross-compilation to different sets of targets. Visual Studio 2015 uses a new project type (.xproj
) as a wrapper for these. This post isn’t about enabling an existing .csproj
or .vbproj
project type (the one most people have been using on “regular”) projects to start cross-compiling. Converting an existing project to use .xproj
is a topic for another day and not all project types are supported by .xproj
.
What this does do is enable the NuGet v3 features to be used by the existing project types today. If you have a .NET 4.6 desktop project, this will not change that. Likewise if your project is using the Xamarin Android 6 SDK, this won’t alter that either. It’ll simply make package management easier.
Acknowledgments
I would like to thank Andrew Arnott for his persistence in figuring out how to make this all work. He explained it to me as he was figuring it out and then recently helped to review this post. Thanks Andrew! A shout out is also due to Scott Dorman and Jason Malinowski for their valuable feedback reviewing this post.
Now that “Packages are now stored in a per-user cache instead of alongside the solution”, does this mean you can no longer store packages in source control as part of your application?
In general, it was never recommended practice to store packages in source control once NuGet gained restore capability. You can change the location of the cache, as per the release notes in 3.2, but that’s a global change, not per solution.
We always store our packages in source control (see http://blog.ploeh.dk/2014/01/29/nuget-package-restore-considered-harmful/), its a shame this option is no longer supported
Once I converted a project to project.json, I started getting build errors for standard .NET types like System.Diagnostics.Stopwatch or System.Collections.Specialized.NameValueCollection. My framework moniker is “net461”. Am I missing anything?
My fault, I didn’t read your post carefully – false did the trick.
It should have been the ‘IncludeFrameworkReferencesFromNuGet’ tag in the previous comment
Can I add comments within project.json though?
Thank you so much for your post. I decided this would be interesting to try with one of my projects. The setup was pretty easy, as you had detailed and Visual Studio seems to build the project just fine. We have a need to build the project from the command line though. When I try to invoke the command(running as Administrator) “C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\devenv.exe” “solution-path.sln” /build Release though, it fails to create the project.json.lock file and fails to build(silently). My machine has the VS Update 1 and the latest nuget which is why it builds from VS, but I can’t figure out why it fails from the command line.
Did you face any issues like this by any chance and/or have an idea for a solution?
Thanks again for the very interesting post.
Hi Eniep,
For command-line builds, you need to call
NuGet.exe restore the.sln
prior to the build step. That will generate theproject.lock.json
files and download any needed packages.Just make sure you’re using at least the 3.3 version of NuGet.exe; my build scripts tend to always download the latest version from the dist link mentioned in my post.
Have you been able to get references to non-
project.json
-based projects working from within aproject.json
project? E.g., suppose I have some ordinary.csproj
-based project that I wish to use from a project I’ve converted to useproject.json
. You can reference such projects from ASP.NET Core (aka 5) projects, but I’ve not been able to get it to work with non-ASP.NET Core projects. You end up having to do an ordinary project reference in the.csproj
which in turns means there’s no automatic transitive dependency handling.As far as I know, there’s no current way to get transitive dependencies working from a non-project.json csproj. That’s a feature of project.json with it’s project.lock.json mechanisms. That said, project.json should work with any csproj project type.
If there’s a way to make this work, I’d be interested to know.
I’ve been using this pattern of references in my full .NET framework projects for some time now and it works great! However, I’ve never got it to work on my build server. I do a package restore and it creates the project.json.lock file, but then MSBuild fails to actually resolve any of the nuget-defined references and hence can’t compile the project. I’ve hit this using Bamboo and TeamCity build systems. Are there some properties I need to pass in to MSBuild to get it to work with project.json for ordinary class library projects?
Nevermind, I found that my build server was missing the necessary targets and props to handle project.json references. Namely Microsoft.NuGet.targets, and the targets that invoke these – Microsoft.Common.targets.
I found my project files all conditionally imported Microsoft.Common.props – if it exists. This is part of the default project templates Visual Studio creates. Well this file didn’t exist on my build server and hence the common targets and quietly were not executed, which are responsible for loading the nuget targets, which are responsible for resolving the project.json references.
Ensuring these were present in the referenced locations made everything work as expected.
NuGet 3.4 is out now.
Any comments on this https://twitter.com/davidfowl/status/730219570783363073 ?
project.json is dead…
I suspect David’s comments relate only to the deprecation of project.json as a replacement for csproj files, and not as a replacement for packages.config. project.json is supported by NuGet and that isn’t going away, so everything in this article still applies.
Note the section in this article “What this doesn’t do” where it mentions the very new (hasn’t hit RTM yet) cross-platform project management stuff. That has been using additional content in the project.json file beyond just NuGet references, and this appears to be what David is saying – project.json will no longer be used this way and will most likely remain only for referencing NuGet packages.
In that thread, when Brandon asked “does this mean nuget 3 is also going to stop using it and go back to sln-level storage & packages.config?” David replied “nope”.
And more generally, in the discussions at https://blogs.msdn.microsoft.com/webdev/2016/05/11/notes-from-the-asp-net-community-standup-may-10-2016/ and https://github.com/aspnet/Home/issues/1433 the message seems to be that the particular features Oren discussed here will not be going away. (Exactly how they’ll be represented doesn’t seem to be nailed down yet, but the important point is that things like not needing to explicitly list the transitive closure of your references aren’t going away just because they’re moving away from project.json.)
You might be right today, but given recent history everything could change tomorrow. I don’t think it’s advisable to “project.json all the things” until some of this dust settles.
This post is a lifesaver.
Had high hopes for that workaround as using csproj projects as reference is no longer possible in RC2. Unfortunately it really disables csproj build or at least ignores it output and takes the output of xproj (which was automatically created for me). Unfortunately there was a reason csproj and that was custom tools for ANTLR grammar files. They stopped being converted as soon as I add project.json and it starts building it into net461 subfolder in bin\Debug. I haven’t tried with T4 but I am afraid it will be the same.
After Jon Allen posted that .NET Core Plans to Drop project.json i am cautious to switch over to project.json,
Great article. Thank you for taking the time to write about this. I have subscribed to your RSS. 🙂
Additionally, I have referenced this post from a StackOverflow question I have here, if you care to answer:
https://oren.codes/2016/02/08/project-json-all-the-things/
Thanks again!
Hi. I’m trying to convert one of my projects to use project.json, adding the following imports: “portable-net45+wpa81+wp8+win8”
The thing is that I’m having a dependency over Rx-Main, and when I build the project I’m getting an ugly error:
“Severity Code Description Project File Line Suppression State
Error CS0012 The type ‘IServiceProvider’ is defined in an assembly that is not referenced. You must add a reference to assembly ‘System.ComponentModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’.”
Is there anything else that I should add to get the project building? I tried adding the code below, but didin’t work.
“frameworkAssemblies”: {
“System.ComponentModel”: “”
},
Thanks,
Mauro
I would recommend using
System.Reactive
version3.0.0
as it’s designed to support .NET Standard.Has anybody written/found a script for accomplishing these steps?
Here is a good article that talks about the process of converting from packages.config to project.json: https://github.com/NuGet/Home/wiki/Converting-a-csproj-from-package.config-to-project.json
That article links to a script that can convert all of the projects in a solution:
https://github.com/wgtmpeters/nugetprojectjson
Awesome article and I’m glad it’s here. It’s made our build process so much better.
One problem I’m running into and that searching has not been able to resolve:
When using the NuGet package manager or package manager console, if the project.json file is not checked out (TFVC), I get an Access Denied error. I can’t be the only one running into this but my google-fu appears to be weak as I can’t find any reference to it. I was wondering if you or anyone else reading this article has run into this problem and if a solution has been found?
Hi Oren,
thanks for your post. I’m curiuos if converting to project.json could help with my scenario:
I’m trying to extract portable portions from our net46 assemblies into portable netstandard 1.3 assemblies. For cases where e.g. ICloneable is needed, I created a bait-and-switch PCL (called A.Core: netstandard 1.3, net46 and uap).
My portable netstandard 1.3 assembly (A.Common.Portable) nugets this PCL in order to implement a cloneable class. Compiles fine.
Another assembly (net46; A.Consumer) references the portable assembly and nugets the PCL, too. ‘References’ show the net46 assembly variant of the PCL (A.Core-net46). At the location where the cloneable class is used, I get ‘The type ‘ICloneable’ is defined in an assembly that is not referenced. You must add a reference to assembly A.Core-Portable, Version=1.0.3.0, Culture=neutral, PublicKeyToken=22ffbf1e563363a8′.’ compiler errors.
That looks like A.Common.Portable uses the PCL version of ICloneable and that is not identical to the net46 version. That switch is made at compile time of A.Common.Portable and is not revised when both are referenced in the A.Consumer net46 assembly.
Is there a solution?
-Thomas
Now i’m not sure to migrate since is announced VS will switch back to .csproj projects again ¬¬