ID Area Title Description 17 area-System.Xml Some XPath.XDocument tests are failing Some XPath.XDocument queries have different results than other XPath navigators. This might be an old behavior or newly introduced bug. Failing tests: build /p=IncludeTraits=ActiveIssue=17 20 area-System.Xml 2 XPath.XDocument tests fail because of lacking feature "XPath.XDocument navigator doesn't support MoveToId(string). Verify if this was ever supported. If it was, verify if we want to support it in the future. If it wasn't move the tests to a different file and remove them from XPath.XDocument.Tests project. Failing tests: NodeSetFunctionsTest2267 MatchesTest2352 " 22 area-System.Numerics Two Numerics Tests are failing only on our CI server Two of the tests in our System.Numerics.Vectors suite are failing only on our CI build server, and potentially only intermittently: Vector2NormalizeTest1 Vector4NormalizeTest2 Given that these are very similar to other tests which cover a similar edge-case (especially the Vector3 normalization tests, which aren't failing), we will need to investigate why these tests in particular are failing on our build server. This may have been a point-in-time issue as we brought up our build infrastructure, and may not re-surface again. 36 area-System.Numerics SIMD test failures on non-ENU configurations. After pulling both of @adamralph 's pull requests #31 and #32, I'm continuing to see test failures for SIMD on a DEU (German) test environment. Here's a representative error: d:\oss\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(545): error : System.Numerics.Tests.GenericVe ctorTests.ToStringCurrencySByte: Assert.Equal() Failure\r\nPosition: First difference is at position 8\r\nExpected: <97,00 ?, -108,00 ?, 22,00 ?, 29,00 ?, 49,00 ?, 60,00 ?, 103,00 ?, 58,00 ?, -62,00 ?, -124,00 ?, -117,00 ?, 48,00 ?, 15,00 ?, -35,00 ?, -13,00 ?, -34,00 ?>\r\nActual: <97,00 ?. -108,00 ?. 22,00 ?. 29,00 ?. 49,00 ?. 60,00 ?. 103 ,00 ?. 58,00 ?. -62,00 ?. -124,00 ?. -117,00 ?. 48,00 ?. 15,00 ?. -35,00 ?. -13,00 ?. -34,00 ?> [D:\oss\corefx\bin\ tools\fxbuild.proj] Observe that expected separates elements with a comma, actual separates elements with a dot. 41 area-System.Numerics Quaternion operator overloads should be using the respective methods Quaternion declares a handful of methods to perform addition, subtraction and multiplication, and provides the respective overloads for these operations. However, instead of re-using the `Add`, `Multiply` etc. methods, the code is re-written in the operator overloads. The operators should be using their respective methods rather than re-declaring the same code. This is under the assumption that the JIT inlines the methods when they are used in the operator overloads. 49 area-Infrastructure Add Linux/Mac build script A `build.sh` should be added alongside `build.cmd` to build corefx on Linux/Mac. 50 area-System.Numerics Made Quarternion's operator overloads use their respective methods "Operator overloads of Quaternion now utilize their respective methods, and removed redundant ""this"" qualifiers in Quaternion constructor. " 52 area-Meta ReferenceSource repo license incorrect for individual files some of the files I looked up, it has header comment with Apache 2.0 license. which license should apply for those files? MIT or Apache 2.0 54 area-System.Xml "Remove always true ""if"" and unreachable code in System.Xml.Linq.XObject.SkipNotify method." "This: if (o.Annotations() != null) is always true because the above while can finish only in two conditions: when o != null or when o.annotations == null. The first condition will be catched by "" if (o == null)"" and if the second one is true, ""o.Annotations() != null"" will also always be true i think. " 55 area-System.Xml [Issue 54] Removed always-true if and unreachable code in XObject.cs Removed always-true if and unreachable code in XObject.cs 58 area-System.Xml System.Xml.sln fails to build on Mono, error CS0433 **I know that cross-platform support is coming later, I just thought it might make sense to document this here in case someone else tries the same** Building System.Xml.sln with xbuild on Mono doesn't work (the other solutions build fine), it throws the following errors: ``` Build FAILED. Errors: /home/alexander/dev/corefx/src/System.Xml.sln (default targets) -> (Build target) -> /home/alexander/dev/corefx/src/System.Xml.XPath.XDocument/System.Xml.XPath.XDocument.csproj (default targets) -> /usr/lib/mono/4.5/Microsoft.CSharp.targets (CoreCompile target) -> System/Xml/XPath/XAttributeExtensions.cs(10,56): error CS0433: The imported type `System.Xml.Linq.XAttribute' is defined multiple times System/Xml/XPath/XAttributeExtensions.cs(10,78): error CS0433: The imported type `System.Xml.Linq.XNamespace' is defined multiple times System/Xml/XPath/XDocumentExtensions.cs(22,61): error CS0433: The imported type `System.Xml.Linq.XNode' is defined multiple times System/Xml/XPath/XObjectExtensions.cs(10,23): error CS0433: The imported type `System.Xml.Linq.XContainer' is defined multiple times System/Xml/XPath/XObjectExtensions.cs(10,49): error CS0433: The imported type `System.Xml.Linq.XObject' is defined multiple times System/Xml/XPath/XNodeNavigator.cs(48,9): error CS0433: The imported type `System.Xml.Linq.XElement' is defined multiple times System/Xml/XPath/XNodeNavigator.cs(784,35): error CS0433: The imported type `System.Xml.Linq.XText' is defined multiple times ``` It looks like it runs into a conflict with the System.Xml.Linq library in the Mono GAC. Needs further investigation. 69 area-Infrastructure build.cmd does not build solution on HP laptop (when Platform=MCD is pre-set) "I've got HP laptop with windows 7. When I run build.cmd, I've got the error ""configuration is invalid"" ``` C:\Projects\dotnet\corefx>build.cmd C:\Projects\dotnet\corefx\src\System.Collections.Immutable.sln.metaproj : error MSB4126: указанная конфигурация решения ""Release|MCD"" недопустима. Укажите допустимую конфигурац ию решения с помощью свойств Configuration и Platform (например, MSBuild.exe Solution.sln /p:Configuration=Debug /p:Platform=""Any CPU"") или оставьте эти свойства пустыми, чтобы использовать конфигурацию решения по умолчанию. [C:\Projects\dotnet\corefx\src\System.Collections.Immutable.sln] ``` The reason of this error is that environment variable `Platform` is set to `MCD` on the laptop by default. I have to change build command in the build.cmd by adding `/p:Platform=""Any CPU""` to compile the project. Build script should check for allowed platforms and generate user-friendly error message, which says what to do to successfully compile the project when the platform is not supported. " 70 area-Meta Hello, World! sample "Feature request: please add ""Hello, World!"" sample, which shows how to use .NET Core with user applications. " 71 area-System.Numerics Behaviour of `Quaternion.CreateFromAxisAngle` when axis is not a unit vector "The resulting quaternion depends on the length of `axis`. This does not describe a rotation and thus does not match the documentation. There are a few ways to handle this issue: 1. Normalize `axis`, but this incurs a performance hit 2. Add a precondition that `axis.Length()` ≈ 1. But enforcing that pre-condition is about as expensive as 1), so it'd probably need to be a documentation-only precondition. Unfortunately this means that consumers will rely on the current behaviour even if it's undocumented, so I don't think that this is a good solution. 3. Document the current behaviour --- Just for convenience, the relevant source code: ``` /// /// Creates a Quaternion from a vector and an angle to rotate about the vector. /// /// The vector to rotate around. /// The angle, in radians, to rotate around the vector. /// The created Quaternion. public static Quaternion CreateFromAxisAngle(Vector3 axis, float angle) { Quaternion ans; float halfAngle = angle * 0.5f; float s = (float)Math.Sin(halfAngle); float c = (float)Math.Cos(halfAngle); ans.X = axis.X * s; ans.Y = axis.Y * s; ans.Z = axis.Z * s; ans.W = c; return ans; } ``` " 72 area-System.Numerics `Equals` with NaN values (IEEE vs. reflexivity) The built in floating point types compare `NaN` as unequal when using `==` and `!=` (following IEEE semantics) but compare it as equal when using the `Equals` method. Your floating point based types currently use IEEE semantics even for `Equals`. I suggest using the same behaviour as the built in types in your floating point based types like vectors or quaternions. The MSDN documentation of `Equals` contains an exception that allows `A.Equals(A)` to return false on floating point types, so you don't strictly violate its contract. But returning false still breaks hash tables and does not match the behaviour of the built in types, so I consider it a bad idea. This can be avoided by calling `Equals` on the members instead of `==` in the implementation of `Equals` but not in `==` and `!=`. For example with quaternion, replace ``` public bool Equals(Quaternion other) { return (X == other.X && Y == other.Y && Z == other.Z && W == other.W); } ``` with ``` public bool Equals(Quaternion other) { return (X.Equals(other.X) && Y.Equals(other.Y) && Z.Equals(other.Z) && W.Equals(other.W)); } ``` You might want to add tests that check that `==` and `!=` compare all the above cases as unequal, so that they match the IEEE specification. Replace: ``` // Counterintuitive result - IEEE rules for NaN comparison are weird! Assert.False(a.Equals(a)); Assert.False(b.Equals(b)); Assert.False(c.Equals(c)); Assert.False(d.Equals(d)); ``` with: ``` // Equals does not follow IEEE semantics since many consumers rely on equality being reflexive. // This includes collections like `Dictionary` or `HashSet` Assert.True(a.Equals(a)); Assert.True(b.Equals(b)); Assert.True(c.Equals(c)); Assert.True(d.Equals(d)); // Counterintuitive result - IEEE rules for NaN comparison are weird! Assert.False(a == a); Assert.False(b == b); Assert.False(c == c); Assert.False(d == d); Assert.True(a != a); Assert.True(b != b); Assert.True(c != c); Assert.True(d != d); ``` 77 area-System.Xml Some Xml encoding tests lost their encoding Some of Xml tests were testing problematic characters in different Encoding (like Russian characters). At some point few years back the encoding was lost while moving between different repos. Tests pass because .cs files lost their encoding too. Tests need to be rewritten since there is no trace of original copy 81 area-System.Xml Add test coverage for XPath and XDocument - Adding test coverage for XPath implementations (XPath.XPathDocument, XPath.XmlDocument, XPath.XDocument) - Adding some tests for XDocument (more coming later) 94 area-Meta Necessary bits for Mono.Posix I tried to build [Mono.Posix](https://github.com/mono/mono/tree/master/mcs/class/Mono.Posix) against aspnetcore50 and found a lot of API that mono uses to be missing from .NET Core most prominently in the Interop-domain (like CustomMarshaler). Please add Mono.Posix to your list of scenarios that can benefit from the API surface of .NET Core. 110 area-System.Xml Add async document/element loading for XLinq. Adds XElement.LoadAsync and XDocument.LoadAsync. Code from the sync versions has been largely lifted out so they can share an implementation as much as possible. 116 area-System.Numerics Quaternion and public fields Is there a reason that the Quaternion struct has public exposed fields versus read only properties? Seems to be against the general guidelines for structs and immutability. Why would you want to allow X, Y, Z and W to be set outside of the ctor? 118 area-System.Numerics Matrix4x4 - more useful public properties Forward, Backward, Up, Down, Left, Right public properties for Matrix4x4 119 area-System.Numerics Vector3 - more public static properties Up, Down, Left, Right, Forward, Backward vector public static properties 121 area-System.Numerics Matrix4x4 - more useful public properties Added Forward, Backward, Up, Down, Left, Right vectors public properties for Matrix4x4. Fix #118 129 area-System.Xml Add XmlReader System.Xml.XmlReader is currently missing. 136 area-System.Xml XPath tests with expressions resulting in +/- infinity fail in Windows 10 "The following XPath tests fail for me on Windows 10 Technical Preview, build 9879: StringFunctionsTest2457 StringFunctionsTest2458 The call to Convert.ChangeType in XPathTests.Common.Utils.XPathObject is throwing a FormatException when the value passed is ""Infinity"" or ""-Infinity"". The tests pass for me on Windows 8.1. Each of these tests is part of 3 different test assemblies, resulting in 6 failures total: System.Xml.XPath.Tests.dll System.Xml.XPath.XDocument.Tests.dll System.Xml.XPath.XmlDocument.Tests.dll " 141 area-System.Numerics Added .ToArray() and .FromArray() for Vector types This commit adds Vector2.ToArray(), Vector2.FromArray(float[]), Vector3.ToArray(), Vector3.FromArray(float[]), Vector4.ToArray() and Vector4.FromArray(float[]). The FromArray method copies the array elements to a vector: the element at index 0 becomes X, the element at index 1 becomes Y and so on. The ToArray() method converts the vector to an array: it puts X at index 0, Y at index 1 and so on. These methods looked useful to me if you have an array but want to do vector operations. 142 area-System.Numerics Vector2 and Vector3 Projection The current source includes public static methods for reflection of a vector about a unit vector normal but the underlying vector projection is also useful by itself. Proposing: ``` C# public static Vector2 Project(Vector2 vector, Vector2 component) { ... } //... public static Vector3 Project(Vector3 vector, Vector3 component) { ... } ``` I'll wait for feed back before trying to submit a pull request. 144 area-Infrastructure All the linked files are in the root folder of the XML projects. When you reference files outside the project directory, such as the common directory in the XML code case, these links by default show up in the root of the project in VS which is very annoying and clutters the solution unnecessarily. We should add the appropriate metadata to these Compile items so that VS will group them correctly in the solution explorer. 149 area-System.Xml Wrong named tests in System.Xml.XmlDocument "- The files NodeChangingTests.cs, NodeChangedTests.cs, NodeInsertingTests.cs and NodeInsertedTests.cs contain test cases called ""RemoveEventHandler"". No test uses a ""Remove""-event. Maybe a Copy&Paste mistake from file NodeRemovedTests.cs. - The files NodeChangingTests.cs and NodeChangedTests.cs contain test cases called ""RemoveNode"" but testing a change of a node. " 156 area-System.Numerics Private static read only fields for Vectors Private static read only fields for One, Zero and Unit vectors on Vector2, Vector3 and Vector4. 157 area-System.Numerics Alternative versions of methods for Vectors and Matrices "Would be helpful to have alternative versions of methods with ""out"" parameter as result. Example: this ```C# public static Matrix CreateFromQuaternion(Quaternion quaternion) { ... } ``` and that ```C# public static void CreateFromQuaternion(ref Quaternion quaternion, out Matrix result) { ... } ``` What do you think guys? Do you want to add that? " 176 area-System.Linq System.Linq.Parallel Tests are Failing on AppVeyor Machines The following 5 tests are failing (some intermittently) on our AppVeyor CI builds. I haven't investigated deeply, but I suspect it has to do with the limited hardware that the VM's are equipped with. Some of the test explicitly test whether multiple threads are created, which might not even be the case in an underpowered VM. 181 area-System.Linq Removes unnecessary object allocations because of delegates "I introduced several singletons to store delegate instances to avoid unnecessary object allocation in generic methods. #### For example `ParallelEnumerable.PerformAggregation()` method **Before:** ``` csharp private static T PerformAggregation(this ParallelQuery source, Func reduce, T seed, bool seedIsSpecified, bool throwIfEmpty, QueryAggregationOptions options) { Contract.Assert(source != null); Contract.Assert(reduce != null); Contract.Assert(options.IsValidQueryAggregationOption(), ""enum is out of range""); AssociativeAggregationOperator op = new AssociativeAggregationOperator( source, seed, null, seedIsSpecified, reduce, reduce, delegate (T obj) { return obj; }, throwIfEmpty, options); return op.Aggregate(); } ``` ``` IL_0000: nop IL_0001: ldarg.0 IL_0002: ldarg.2 IL_0003: ldnull IL_0004: ldarg.3 IL_0005: ldarg.1 IL_0006: ldarg.1 IL_0007: ldnull IL_0008: ldftn !!0 System.Linq.ParallelEnumerable::'b__e'(!!0) >>> IL_000e: newobj instance void class [System.Runtime]System.Func`2::.ctor(object, native int) IL_0013: ldarg.s throwIfEmpty IL_0015: ldarg.s options IL_0017: newobj instance void class System.Linq.Parallel.AssociativeAggregationOperator`3::.ctor(class [System.Runtime]System.Collections.Generic.IEnumerable`1, !1, class [System.Runtime]System.Func`1, bool, class [System.Runtime]System.Func`3, class [System.Runtime]System.Func`3, class [System.Runtime]System.Func`2, bool, valuetype System.Linq.Parallel.QueryAggregationOptions) IL_001c: stloc.0 IL_001d: ldloc.0 IL_001e: callvirt instance !2 class System.Linq.Parallel.AssociativeAggregationOperator`3::Aggregate() IL_0023: stloc.1 IL_0024: br.s IL_0026 IL_0026: ldloc.1 IL_0027: ret ``` **After:** ``` csharp private static T PerformAggregation(this ParallelQuery source, Func reduce, T seed, bool seedIsSpecified, bool throwIfEmpty, QueryAggregationOptions options) { Contract.Assert(source != null); Contract.Assert(reduce != null); Contract.Assert(options.IsValidQueryAggregationOption(), ""enum is out of range""); AssociativeAggregationOperator op = new AssociativeAggregationOperator( source, seed, null, seedIsSpecified, reduce, reduce, IdentityFunction.Instance, throwIfEmpty, options); return op.Aggregate(); } ``` ``` IL_0000: nop IL_0001: ldarg.0 IL_0002: ldarg.2 IL_0003: ldnull IL_0004: ldarg.3 IL_0005: ldarg.1 IL_0006: ldarg.1 >>> IL_0007: ldsfld class [System.Runtime]System.Func`2 class System.Linq.IdentityFunction`1::Instance IL_000c: ldarg.s throwIfEmpty IL_000e: ldarg.s options IL_0010: newobj instance void class System.Linq.Parallel.AssociativeAggregationOperator`3::.ctor(class [System.Runtime]System.Collections.Generic.IEnumerable`1, !1, class [System.Runtime]System.Func`1, bool, class [System.Runtime]System.Func`3, class [System.Runtime]System.Func`3, class [System.Runtime]System.Func`2, bool, valuetype System.Linq.Parallel.QueryAggregationOptions) IL_0015: stloc.0 IL_0016: ldloc.0 IL_0017: callvirt instance !2 class System.Linq.Parallel.AssociativeAggregationOperator`3::Aggregate() IL_001c: stloc.1 IL_001d: br.s IL_001f IL_001f: ldloc.1 IL_0020: ret ``` Where `IdentityFunction` looks like ``` csharp internal class IdentityFunction { public static readonly Func Instance = Function; private static T Function(T arg) { return arg; } } ``` " 187 area-System.IO Can the System.IO namespace be modified to manipulate device handles? "This assumes the `System.IO` and 'Microsoft.Win32.SafeHandles' namespace are being open sourced in CoreFX. If not close this ticket. Right now if you do the following: ``` var handle = File.Open(""\\\\.\\Global\\ProcmonDebugLogger"", FileMode.Append); ``` You get an exception **FileStream will not open Win32 devices such as disk partitions and tape drives. Avoid use of ""\.\"" in the path.** If you want raw device access, you need to use P/Invoke. I propose the following: ``` public sealed class Microsoft.Win32.SafeHandles.SafeDeviceHandle: SafeHandleZeroOrMinusOneIsInvalid { . . . } public static Microsoft.Win32.SafeHandles.SafeFileHandle System.IO.File.OpenHandle(string path) { public bool DeviceIoControl( . . .) { . . . } } public static Microsoft.Win32.SafeHandles.SafeDeviceHandle System.IO.File.OpenDeviceHandle(string path) { . . .} ``` My use case for this would be to remove the [PInvoke functons from ProceMonDebugOutput](https://github.com/Wintellect/ProcMonDebugOutput/blob/master/Source/Sysinternals.Debug/NativeMethods.cs) by @JohnWintellect. " 195 area-System.Xml [System.Xml.XPath] Few tests fail sometimes fail because of NRE Few tests are failing in the same place because of NRE. Stack trace (RegressionTestsTest557): E:\oss\corefx\bin\tools\tests.targets(17,5): error : XPathTests.FunctionalTests.MiscellaneousCases.RegressionTestsTests.RegressionTestsTest557: System.NullReferenceException : Object reference not set to an instance of an object. [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XmlDocumentEx.GetNamespaceXml(XmlDocument xmlDocument)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.DocumentXPathNavigator.MoveToFirstNamespace(XPathNamespaceScope scope)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.NamespaceQuery.Advance()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.Query.MoveNext()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.Query.get_Count()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.XPathSelectionIterator.get_Count()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.XPathNodesetTest(String xml, String testExpression, XPathResult expected, XmlNamespaceManager namespaceManager, String startingNodePath)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.FunctionalTests.MiscellaneousCases.RegressionTestsTests.RegressionTestsTest557() [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : XPathTests.FunctionalTests.CoreFunctionLibrary.StringFunctionsTests.StringFunctionsTest2465: System.NullReferenceException : Object reference not set to an instance of an object. [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XmlDocumentEx.GetNamespaceXml(XmlDocument xmlDocument)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.DocumentXPathNavigator.MoveToNextNamespace(XPathNamespaceScope scope)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.NamespaceQuery.Advance()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.ForwardPositionQuery.Evaluate(XPathNodeIterator context)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.BaseAxisQuery.Evaluate(XPathNodeIterator nodeIterator)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.MergeFilterQuery.Evaluate(XPathNodeIterator nodeIterator)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr, XPathNodeIterator context)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Select(XPathExpression expr)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.CreateNavigator(String xml, String startingNodePath, XmlNamespaceManager namespaceManager)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.XPathObject[T](String xml, String testExpression, XmlNamespaceManager namespaceManager, String startingNodePath)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.XPathStringTest(String xml, String testExpression, Object expected, XmlNamespaceManager namespaceManager, String startingNodePath)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.FunctionalTests.CoreFunctionLibrary.StringFunctionsTests.StringFunctionsTest2465() [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : XPathTests.FunctionalTests.Location.Paths.Axes.ComplexExpressionsTests.ComplexExpressionsTest348: System.NullReferenceException : Object reference not set to an instance of an object. [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XmlDocumentEx.GetNamespaceXml(XmlDocument xmlDocument)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.DocumentXPathNavigator.MoveToNextNamespace(XPathNamespaceScope scope)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.NamespaceQuery.Advance()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.Query.MoveNext()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.Query.get_Count()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.XPathSelectionIterator.get_Count()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.XPathNodesetTest(String xml, String testExpression, XPathResult expected, XmlNamespaceManager namespaceManager, String startingNodePath)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.FunctionalTests.Location.Paths.Axes.ComplexExpressionsTests.ComplexExpressionsTest348() [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : XPathTests.FunctionalTests.Location.Paths.Predicates.UsingPositionFunctionTests.UsingPositionFunctionTest681: System.NullReferenceException : Object reference not set to an instance of an object. [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XmlDocumentEx.GetNamespaceXml(XmlDocument xmlDocument)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.DocumentXPathNavigator.MoveToNextNamespace(XPathNamespaceScope scope)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.NamespaceQuery.Advance()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.MergeFilterQuery.Evaluate(XPathNodeIterator nodeIterator)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr, XPathNodeIterator context)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Select(XPathExpression expr)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.XPathNodesetTest(String xml, String testExpression, XPathResult expected, XmlNamespaceManager namespaceManager, String startingNodePath)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.FunctionalTests.Location.Paths.Predicates.UsingPositionFunctionTests.UsingPositionFunctionTest681() [E:\oss\corefx\bin\tools\fxbuild.proj] Stack trace (NodeSetFunctionsTest22100): E:\oss\corefx\bin\tools\tests.targets(17,5): error : XPathTests.FunctionalTests.CoreFunctionLibrary.NodeSetFunctionsTests.NodeSetFunctionsTest22100: System.NullReferenceException : Object reference not set to an instance of an object. [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XmlDocumentEx.GetNamespaceXml(XmlDocument xmlDocument)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.DocumentXPathNavigator.MoveToNextNamespace(XPathNamespaceScope scope)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.NamespaceQuery.Advance()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.FilterQuery.Advance()\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at MS.Internal.Xml.XPath.MergeFilterQuery.Evaluate(XPathNodeIterator nodeIterator)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr, XPathNodeIterator context)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at System.Xml.XPath.XPathNavigator.Select(XPathExpression expr)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.CreateNavigator(String xml, String startingNodePath, XmlNamespaceManager namespaceManager)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.XPathObject[T](String xml, String testExpression, XmlNamespaceManager namespaceManager, String startingNodePath)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.Common.Utils.XPathStringTest(String xml, String testExpression, Object expected, XmlNamespaceManager namespaceManager, String startingNodePath)\r [E:\oss\corefx\bin\tools\fxbuild.proj] E:\oss\corefx\bin\tools\tests.targets(17,5): error : at XPathTests.FunctionalTests.CoreFunctionLibrary.NodeSetFunctionsTests.NodeSetFunctionsTest22100() [E:\oss\corefx\bin\tools\fxbuild.proj] 201 area-System.Numerics Vector2/Vector3 could implement operator overloads For the sake of intuitive use, Vector2 and Vector3's `Public operator methods` could be extended with actual operator overloads. This promotes intuitive usage where `var vec = vecA * vecB;` is assumed to be more programmer-friendly than `var vec = Vector3.Multiply(vecA, vecB);`. I'm looking forward to your thoughts. 218 area-System.Console Add tests for System.Console library 219 area-System.Runtime Consider adding API to return common Funcs and Actions without reallocating See pull request #181: In that case, caching was proposed for System.Linq.Parallel. This means that other libraries can't share the same cache (which can grow by the number of generic instantiations). Furthermore, since the actual call sites affected in PLINQ were already allocating considerably, it is doubtful that PLINQ would benefit from the optimization. We should study if we can design public API in the BCL in the same vein as the new `Array.Empty().` _Before_ we do this, we should: 1. Gather data about how many places we have across the stack that would benefit from these shared allocations and find use cases in more compelling places than PLINQ. 2. Design the API. We can use this issue to discuss and evolve it in to a mini-spec. Here's an idea just to get that ball rolling on that. ``` C# namespace System { public static class CommonAction { public static Action Empty { get; } } public static class CommonFunc { public static Func AlwaysTrue(); public static Func Identity(); public static Func IsInstanceOf(); public static Func Cast(); } } ``` 220 area-System.Console Fix /property:OS=Unix build of System.Console Some resource strings got lost in the move to GitHub. There was also an inconsistency with the available NuGet packages and where SafeHandle lives. 221 area-System.Numerics Fix vector tests warnings about obsolete Marshal.SizeOf The System.Numerics.Vectors tests were causing build warnings like the following: ``` C# Vector2Tests.cs(15,29): warning CS0618: 'System.Runtime.InteropServices.Marshal.SizeOf(System.Type)' is obsolete: 'SizeOf(Type) may be unavailable in future releases. Instead, use SizeOf(). ``` This changes does exactly that. 224 area-System.Console Implement simple console foreground/background color scoping via IDisposable "Just a simple API evolution that allows the developer to ""scope"" his color updates. Can be used like that : ``` csharp static void Main(string[] args) { Console.WriteLine(""Default""); using (Console.UseBackgroundColor(ConsoleColor.Red)) { Console.WriteLine(""Background red""); using (Console.UseForegroundColor(ConsoleColor.Green)) { Console.WriteLine(""Foreground green""); using (Console.UseForegroundColor(ConsoleColor.Yellow)) { Console.WriteLine(""Foreground yellow""); } Console.WriteLine(""Foreground green""); using (Console.UseBackgroundColor(ConsoleColor.Blue)) { Console.WriteLine(""Background blue""); } Console.WriteLine(""Background red""); } } Console.WriteLine(""Default""); } ``` And outputs like that : ![image](https://cloud.githubusercontent.com/assets/1341236/5394321/69d6a080-813c-11e4-95e9-98856de8555a.png) " 227 area-System.Text Added System.Text.RegularExpressions source This adds the product source code for the System.Text.RegularExpressions library. The tests will require the CoreCLR test runner to execute, so those will be added once the runner is available. 228 area-System.Xml Some tests are failing because of lacking System.String.Format overload build /p=IncludeTraits=ActiveIssue=228 Example tests Test Name: NameWithWhitespace Test FullName: XmlDocumentTests.XmlDocumentTests.CreateElementTests.NameWithWhitespace Test Source: e:\oss\corefx\src\System.Xml.XmlDocument\tests\XmlDocumentTests\CreateElementTests.cs : line 90 Test Outcome: Failed Test Duration: 0:00:00.091 Result Message: Assert.Throws() Failure Expected: typeof(System.Xml.XmlException) Actual: typeof(System.MissingMethodException): Method not found: 'System.String System.String.Format(System.IFormatProvider, System.String, System.Object)'. Result StackTrace: at System.Xml.XmlExceptionHelper.BuildCharExceptionArgs(Char invChar, Char nextChar) at System.Xml.XmlExceptionHelper.BuildCharExceptionArgs(String data, Int32 invCharIndex) in e:\oss\corefx\src\Common\src\System\Xml\ValidateNames.cs:line 349 at System.Xml.XmlDocument.CheckName(String name) in e:\oss\corefx\src\System.Xml.XmlDocument\src\System\Xml\Dom\XmlDocument.cs:line 110 at System.Xml.XmlElement..ctor(XmlName name, Boolean empty, XmlDocument doc) in e:\oss\corefx\src\System.Xml.XmlDocument\src\System\Xml\Dom\XmlElement.cs:line 22 at System.Xml.XmlDocument.CreateElement(String prefix, String localName, String namespaceURI) in e:\oss\corefx\src\System.Xml.XmlDocument\src\System\Xml\Dom\XmlDocument.cs:line 713 at System.Xml.XmlDocument.CreateElement(String name) in e:\oss\corefx\src\System.Xml.XmlDocument\src\System\Xml\Dom\XmlDocument.cs:line 492 at XmlDocumentTests.XmlDocumentTests.CreateElementTests.<>c__DisplayClass10.b__c() in e:\oss\corefx\src\System.Xml.XmlDocument\tests\XmlDocumentTests\CreateElementTests.cs:line 92 230 area-Meta XML Documentation comments not included "I was looking through the recently added Regex code and noticed that a lot of XML documentation summaries are not there, but they do show up in the .NET Framework documentation. Are the comments stored in an external file? If so, isn't it better to include them in the code or provide the external files? Some people would like to compile the libraries themselves and they would be missing a lot of inline documentation. **Example** _.NET Framework_ ![Documentation](http://i.imgur.com/ySqyI81.png) _.NET Core_ ![No Documentation](http://i.imgur.com/RvBK6fh.png) " 231 area-System.Text Improve performance of Regex ctor and IsMatch "The Regex class maintains a cache of byte codes, which the Regex ctor indexes into using a key. It uses this seemingly innocuous line to create that key: ``` C# String key = ((int)options).ToString(NumberFormatInfo.InvariantInfo) + "":"" + cultureKey + "":"" + pattern; ``` This, however, has the unfortunate effect of allocating a string for the options, a string array for the five strings to be passed to the String.Concat call generated by the compiler, another string array allocation inside of Concat, and then the resulting string for the whole operation. The cost of those allocations is causing a non-trivial slowdown for repeated Regex.IsMatch calls for simple regular expressions, such as for a phone number (e.g. from the MSDN docs ""^\d{3}-\d{3}-\d{4}$""). This commit adds a new struct key type that just stores the constitutent options, cultureKey, and pattern, rather than creating a string to store them. That key is then what's stored in each entry in the cache. For repeated Regex.IsMatch calls for basic regular expressions like the phone number one previously mentioned, on my machine this improves throughput by ~35%, in large part due to an ~80% reduction in number of allocations, and (for this particular test case) an ~70% reduction in number of bytes allocated (it depends primarily on the length of the pattern and the length of the culture name). " 234 area-System.Numerics Some System.Numerics.Vector tests are disabled without comment build /p:IncludeTraits=ActiveIssue=234 235 area-System.Linq Some System.Linq.Parallel tests are disabled without comment build /p:IncludeTraits=ActiveIssue=235 240 area-System.Linq Some System.Linq.Parallel tests are super slow on CI Deadlock or super slow build /p:IncludeTraits=ActiveIssue=240 241 area-System.Linq Fix multiple projects for optimizations, debug/trace, etc. System.Linq.Parallel.csproj wasn't outputting symbols, and it didn't have optimizations enabled in Release. System.Numerics.Vectors.csproj was setting TRACE in Release builds but not in Debug builds. System.Text.RegularExpressions.csproj wasn't setting either DEBUG nor RELEASE constants in either Debug/Release builds, it didn't have optimizations enabled in Release, and it wasn't generating symbols in either. The XML libraries were't setting DEBUG/TRACE in either Debug/Release builds, weren't enabling optimizations in Release builds, and weren't generating symbols in either. DEBUG is important in almost all of these libraries, as they use Debug.\* for asserts and the like. Symbol generation is important for debugging. Optimizations are obviously important (but just to verify I saw significant performance wins in multiple libraries just from turning this on in release). TRACE isn't really important, as none of the libraries do release tracing, but I thought it good hygiene to enable it. 247 area-Meta Add Security transparency Since Code Access Security is a thing of the past with .NET Core, I wouldn't expect permissions or security attributes to be used (or even to exist). I didn't see permissions anywhere in the current code: they seem to have be completely removed from .NET Core as expected. However, [several committed files](https://github.com/dotnet/corefx/search?q=SecurityCritical+OR+SecuritySafeCritical&type=Code) still use `SecurityCriticalAttribute` or `SecuritySafeCriticalAttribute`. Is this a left-over from the .NET framework code? I expect so, since they mean nothing without permission checks. If this is intended, what are the guidelines about using them? 249 area-System.Collections Memory allocation documentation for ImmutableDictionary Keys and Values ## Background As described in #246 (and previously in #147), the `Keys` and `Values` properties of `ImmutableDictionary` cannot be used without incurring memory allocations on the heap. However, a slight modification of user code provides _equivalent_ behavior without these allocations. With allocations: ``` csharp foreach (var key in dictionary.Keys) { ... } ``` Without allocations: ``` csharp foreach (var pair in dictionary) { var key = pair.Key; } ``` ## Suggestion Since updating the `Keys` and `Values` properties to eliminate the allocations would be a breaking change (#246), the only recourse is to update the documentation for these properties to describe steps a user can take to avoid allocations if these properties are used heavily within a particular application. ## Caveat Care must be taken to ensure that the documentation does not read in a way that suggests the `Keys` and `Values` properties should not be used. The properties behave properly and would not be even a measurable source of time spent in the application in nearly all cases. This notice is only meant for users making heavy use of these properties who are also actively working to reduce the memory allocations performed within an especially performance-sensitive section of code. 251 area-System.Text Fix DEBUG constant name It looks like DBG was being used instead of DEBUG at System.Text.RegularExpressions. Was this on purpose? 252 area-System.Text Fix some more XML docs I know that the documentation will be updated eventually, but in the meanwhile we can have a few less compiler warnings if you guys agree. 253 area-System.Text Fix GetGroup never returning EmptyGroup This is a very obvious bug: _captureMap probably used to be a hashtable and the code was not properly refactored. I took the opportunity to redo the method using the Dictionary equivalent logic. Not adding unit tests for this, because the unit test project seems to be missing, and I'm not sure if you guys are going to add it or not, so let me know how to proceed. 254 area-System.Text Improve Dictionary access Just spotted this quick dictionary access optimization. 255 area-System.Text Improve Dictionary access 262 area-System.Reflection System.Reflection.Metadata should provide easier signature reading API **EDIT** 12/3/2015 - Replaced entire description with the detailed proposal matching PR #4809 Today, System.Reflection.Metadata provides low-level access to ECMA-335 CLI metadata, but only provides signatures as blobs that must be parsed with direct knowledge of the format as described in the section II.23.2 of the CLI specification. There is a `BlobReader` for reading various elements out of blobs, but it is up to the caller to read the right things at the right positions. This was by-design as the library is designed to sit at the lowest level behind the scenes of higher level API such as Reflection proper, Roslyn, or CCI. The challenge with signatures is that they are variable-length and encode tree structures, and each higher level model that could sit on top of System.Reflection.Metadata will have different representations for the trees. We do not want to introduce an API to build a fully-formed tree that then has to be traversed and rewritten to match the actual use case. This proposal is therefore a middle ground between: 1. Here are the bytes and some helpers. Read the spec to decode. (Status quo) 2. Yet another high-level metadata API. (Out-of-scope for this layer) It works as follows: 1. The caller chooses an arbitrary representation, `TType`, for type symbols and implements `ISignatureTypeProvider`. (See full API spec below). 2. Signatures are parsed by recursive descent and the provider is called to create new type nodes: - _Give me the `TType` that represents this primitive_ - _Give me the `TType` that represents this TypeDefinition_ - _Give me the `TType` that represents an array of this other TType_ - _etc._ # Sample Usage Given a suitable `TypeSymbol` and `TypeSymbolProvider : ISignatureTypeProvider`, here is code walking all of the `TypeSymbol`'s for every field, parameter, and return type: ``` C# using (var stream = File.OpenRead(pathToDll)) using (var peReader = new PEReader(stream)) { var reader = peReader.GetMetadataReader(); var provider = new TypeSymbolProvider(); foreach (TypeDefinitionHandle typeHandle in reader.TypeDefinitions) { TypeDefinition type = reader.GetTypeDefinition(typeHandle); foreach (FieldDefinitionHandle fieldHandle in type.GetFields()) { FieldDefinition field = reader.GetFieldDefinition(fieldHandle); TypeSymbol fieldType = field.DecodeSignature(provider); // ... } foreach (MethodDefinitionHandle methodHandle in type.GetMethods()) { MethodDefinition method = reader.GetMethodDefinition(methodHandle); MethodSignature methodSig = method.DecodeSignature(provider); TypeSymbol returnType = methodSig.ReturnType; // ... foreach (TypeSymbol parameterType in methodSig.ParameterTypes) { // ... } } } } ``` # Full API ## Additions to existing types These provide convenience entry points. There are other use cases where you want to parse only part of a signature or a signature that you did not obtain from the metadata reader. For that, the `SignatureDecoder` ``` C# namespace System.Reflection.Metadata { public struct FieldDefinition { public TType DecodeSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); } public struct MemberReference { public TType DecodeFieldSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); public MethodSignature DecodeMethodSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); // MemberReferenceKind GetKind(); already exists } public struct MethodDefinition { public MethodSignature DecodeSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); } public struct MethodSpecification { public ImmutableArray DecodeSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); } public struct PropertyDefinition { public MethodSignature DecodeSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); } public struct StandaloneSignature { public ImmutableArray DecodeLocalSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); public MethodSignature DecodeMethodSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); public StandaloneSignatureKind GetKind(); } public enum StandaloneSignatureKind { LocalVariables = 1, Method = 0, } public struct TypeSpecification { public TType DecodeSignature( ISignatureTypeProvider provider, SignatureDecoderOptions options=SignatureDecoderOptions.None); } } ``` ## New types ``` C# namespace System.Reflection.Metadata.Decoding { public struct ArrayShape { public ArrayShape(int rank, ImmutableArray sizes, ImmutableArray lowerBounds); public ImmutableArray LowerBounds { get; } public int Rank { get; } public ImmutableArray Sizes { get; } } public interface IConstructedTypeProvider : ISZArrayTypeProvider { TType GetArrayType(TType elementType, ArrayShape shape); TType GetByReferenceType(TType elementType); TType GetGenericInstance(TType genericType, ImmutableArray typeArguments); TType GetPointerType(TType elementType); } public interface IPrimitiveTypeProvider { TType GetPrimitiveType(PrimitiveTypeCode typeCode); } public interface ITypeProvider { TType GetTypeFromDefinition( MetadataReader reader, TypeDefinitionHandle handle, SignatureTypeHandleCode code); TType GetTypeFromReference( MetadataReader reader, TypeReferenceHandle handle, SignatureTypeHandleCode code); } public interface ISignatureTypeProvider : IConstructedTypeProvider, IPrimitiveTypeProvider, ITypeProvider { TType GetFunctionPointerType(MethodSignature signature); TType GetGenericMethodParameter(int index); TType GetGenericTypeParameter(int index); TType GetModifiedType( MetadataReader reader, bool isRequired, EntityHandle modifierTypeHandle, TType unmodifiedType); TType GetPinnedType(TType elementType); } public interface ISZArrayTypeProvider { TType GetSZArrayType(TType elementType); } public struct MethodSignature { public MethodSignature( SignatureHeader header, TType returnType, int requiredParameterCount, int genericParameterCount, ImmutableArray parameterTypes); public int GenericParameterCount { get; } public SignatureHeader Header { get; } public ImmutableArray ParameterTypes { get; } public int RequiredParameterCount { get; } public TType ReturnType { get; } } public enum PrimitiveTypeCode : byte { Boolean = (byte)2, Byte = (byte)5, Char = (byte)3, Double = (byte)13, Int16 = (byte)6, Int32 = (byte)8, Int64 = (byte)10, IntPtr = (byte)24, Object = (byte)28, SByte = (byte)4, Single = (byte)12, String = (byte)14, TypedReference = (byte)22, UInt16 = (byte)7, UInt32 = (byte)9, UInt64 = (byte)11, UIntPtr = (byte)25, Void = (byte)1, } public struct SignatureDecoder { public SignatureDecoder( ISignatureTypeProvider provider, MetadataReader metadataReader=null, SignatureDecoderOptions options=SignatureDecoderOptions.None); public TType DecodeFieldSignature(ref BlobReader blobReader); public ImmutableArray DecodeLocalSignature(ref BlobReader blobReader); public MethodSignature DecodeMethodSignature(ref BlobReader blobReader); public ImmutableArray DecodeMethodSpecificationSignature(ref BlobReader blobReader); public TType DecodeType(ref BlobReader blobReader); } public enum SignatureDecoderOptions { DifferentiateClassAndValueTypes = 1, None = 0, } public enum SignatureTypeHandleCode : byte { Class = (byte)18, Unresolved = (byte)0, ValueType = (byte)17, } } ``` # Notes - The interface segregation of `ISignatureTypeProvider` is for future `TypeNameParser` and `CustomAttributeDecoder` (still under development in dev/metadata branch), which share some but not all of the same requirements for a type provider as the `SignatureDecoder` 267 area-System.Text Remove superfluous internal access modifier I'm not sure I see the point in having the internal modifier on these fields, but if I'm missing something feel free to close. 271 area-System.Text Regex collections should implement generic collection interfaces "`CaptureCollection`, `GroupCollection`, and `MatchCollection` currently only implement the non-generic `ICollection` interface. These collections should implement the generic collection interfaces to better interoperate with more modern APIs, such as LINQ. Since these collections are already indexable, they should implement `IList` and `IReadOnlyList`, as well as the non-generic `IList` (to be consistent with the generic interfaces). ## Rationale and Usage This is certainly a nice-to-have, but it _is_ a long-standing [request](https://connect.microsoft.com/VisualStudio/feedback/details/469250/capturecollection-groupcollection-and-matchcollection-do-not-implement-icollection-t-and-ienumerable-t) that [developers](http://stackoverflow.com/q/7274422) [still](http://jimlynn.wordpress.com/2009/11/04/why-cant-i-use-linq-on-a-capturecollection/) [ask](http://stackoverflow.com/q/16379433) [about](http://stackoverflow.com/q/4108956). Implementing the generic interfaces will allow these collections to be used more easily with LINQ and interoperate better with more modern framework and library APIs. For example, to use these collections with LINQ right now you have to know about and remember to use `Enumerable.Cast()` to cast the non-generic `IEnumerable` into an `IEnumerable`: ``` c# var captures = from capture in match.Groups.Cast().Last().Captures.Cast() select capture.Value; ``` With these changes you'd no longer have to do that: ``` c# var captures = from capture in match.Groups.Last().Captures select capture.Value; ``` Plus, in the above example, you'd get a performance improvement when using `Enumerable.Last.()` as its implementation has a fast-path for collections that implement `IList`. ## Proposed API ``` c# // DebuggerDisplay and DebuggerTypeProxy added [DebuggerDisplay(""Count = {Count}"")] [DebuggerTypeProxy(typeof(RegexCollectionDebuggerProxy))] // Previously only implemented ICollection public class CaptureCollection : IList, IReadOnlyList, IList { // Existing members public int Count { get; } public Capture this[int i] { get; } public IEnumerator GetEnumerator(); object ICollection.SyncRoot { get; } bool ICollection.IsSynchronized { get; } void ICollection.CopyTo(Array array, int arrayIndex); // Proposed members public void CopyTo(Capture[] array, int arrayIndex); IEnumerator IEnumerable.GetEnumerator(); int IList.IndexOf(Capture item); void IList.Insert(int index, Capture item); void IList.RemoveAt(int index); Capture IList.this[int index] { get; set; } void ICollection.Add(Capture item); void ICollection.Clear(); bool ICollection.Contains(Capture item); bool ICollection.IsReadOnly { get; } bool ICollection.Remove(Capture item); int IList.Add(object value); void IList.Clear(); bool IList.Contains(object value); int IList.IndexOf(object value); IList.Insert(int index, object value); bool IList.IsFixedSize { get; } bool IList.IsReadOnly { get; } void IList.Remove(object value); void IList.RemoveAt(int index); object IList.this[int index] { get; set; } } // DebuggerDisplay and DebuggerTypeProxy added [DebuggerDisplay(""Count = {Count}"")] [DebuggerTypeProxy(typeof(RegexCollectionDebuggerProxy))] // Previously only implemented ICollection public class GroupCollection : IList, IReadOnlyList, IList { // Existing members public int Count { get; } public Group this[int groupnum] { get; } public Group this[String groupname] { get; } public IEnumerator GetEnumerator(); object ICollection.SyncRoot { get; } bool ICollection.IsSynchronized { get; } void ICollection.CopyTo(Array array, int arrayIndex); // Proposed members public void CopyTo(Group[] array, int arrayIndex); IEnumerator IEnumerable.GetEnumerator(); int IList.IndexOf(Group item); void IList.Insert(int index, Group item); void IList.RemoveAt(int index); Group IList.this[int index] { get; set; } void ICollection.Add(Group item); void ICollection.Clear(); bool ICollection.Contains(Group item); bool ICollection.IsReadOnly { get; } bool ICollection.Remove(Group item); int IList.Add(object value); void IList.Clear(); bool IList.Contains(object value); int IList.IndexOf(object value); IList.Insert(int index, object value); bool IList.IsFixedSize { get; } bool IList.IsReadOnly { get; } void IList.Remove(object value); void IList.RemoveAt(int index); object IList.this[int index] { get; set; } } // DebuggerDisplay and DebuggerTypeProxy added [DebuggerDisplay(""Count = {Count}"")] [DebuggerTypeProxy(typeof(RegexCollectionDebuggerProxy))] // Previously only implemented ICollection public class MatchCollection : IList, IReadOnlyList, IList { // Existing members public int Count { get; } public virtual Match this[int i] { get; } public IEnumerator GetEnumerator(); object ICollection.SyncRoot { get; } bool ICollection.IsSynchronized { get; } void ICollection.CopyTo(Array array, int arrayIndex); // Proposed members public void CopyTo(Match[] array, int arrayIndex); IEnumerator IEnumerable.GetEnumerator(); int IList.IndexOf(Match item); void IList.Insert(int index, Match item); void IList.RemoveAt(int index); Match IList.this[int index] { get; set; } void ICollection.Add(Match item); void ICollection.Clear(); bool ICollection.Contains(Match item); bool ICollection.IsReadOnly { get; } bool ICollection.Remove(Match item); int IList.Add(object value); void IList.Clear(); bool IList.Contains(object value); int IList.IndexOf(object value); IList.Insert(int index, object value); bool IList.IsFixedSize { get; } bool IList.IsReadOnly { get; } void IList.Remove(object value); void IList.RemoveAt(int index); object IList.this[int index] { get; set; } } ``` ## Details - There was some discussion as to whether only the read-only interfaces should be implemented, or both the read-only and mutable interfaces. The consensus is to implement both the read-only and mutable interfaces. This is consistent with other collections in the framework. The mutable interfaces are implemented as read-only: mutable members are implemented explicitly and throw `NotSupportedException` (like `ReadOnlyCollection`). - There was an open question as to whether the non-generic `IList` should be implemented as well. These collections are indexable and if `IList` and `IReadOnlyList` are being implemented, `IList` should be implemented as well. This does add several more members, but they are all implemented explicitly so they don't add any new public members to intellisense, and the implementations are very straightforward. - `ICollection.CopyTo` is implemented implicitly (public). - All other new members are implemented explicitly (non-public): - Mutable members are implemented explicitly because these collections are read-only and the mutable members throw `NotSupportedException` (like `ReadOnlyCollection`). - `IList` members are implemented explicitly to hide non-generic members from intellisense. - `IList.IndexOf` and `ICollection.Contains` are implemented explicitly because these methods aren't very useful for these collections and should not be visible in intellisense by default. They're not useful because an implementation using `EqualityComparer.Default` (consistent with other collections) will search the collection using reference equality due to the fact that `Capture`, `Group`, and `Match` do not implement `IEquatable` and do not override `Equals()` and `GetHashCode()`. Further, these types do not have public constructors -- they are created internally by the regex engine, making it very unlikely that you'd want to search for an item in a collection ""A"" that was obtained from collection ""B"". - `IEnumerable.GetEnumerator()` must be implemented explicitly because the non-generic `IEnumerable.GetEnumerator()` is already implemented implicitly and we can't overload on return type. This also precludes returning a `struct` `Enumerator` (for better enumeration performance) because changing the return type of the existing method would be a binary breaking change. As a result, you'll still have to specify the type when using `foreach` (e.g. `foreach (Capture capture in captures)`); you won't be able to use `var` (e.g. `foreach (var capture in captures)`), unfortunately. ## Open Questions - Should `GroupCollection` implement `IDictionary`, `IReadOnlyDictionary`, and `IDictionary`? `GroupCollection` already has a string indexer. Is it worth implementing the dictionary interfaces as part of this? Personally, I'm leaning toward ""no"" because there isn't a compelling scenario for the dictionary interfaces, and they can always be added in the future when needed. ## Pull Request A PR with the proposed changes is available: #1756 ## Updates - Edited this description to make it more of a _speclet_, based on the discussion below and the proposed API Review process. - Some improvements based on feedback from @sharwell. - Fixed existing members. - Added `IList`. These collections are indexable and it would be strange if `IList` and `IReadOnlyList` were implemented alongside `ICollection` but without `IList`. - Added `DebuggerDisplay` and `DebuggerTypeProxy` attributes. - Made `ICollection.CopyTo` implicit (public). " 277 area-System.Text Update regular expressions collections to implement IReadOnlyList Fixes #271 I believe this is the least intrusive path to implementing the requested functionality. I considered the following items but decided to not include them with this pull request. - `GroupEnumerator` is internal, so it might be possible to change the type of `GroupEnumerator.Capture` from `Capture` to `Group`. This would make the explicit cast in the generic `Current` property unnecessary. - I did not update `CaptureCollection` to implement `IReadOnlyDictionary`, primarily because it's a larger change with the addition of the `Keys` and `Values` properties (even if they are explicitly implemented). This should probably be reviewed independently from a straightforward implementation of `IReadOnlyList`. 281 area-System.Diagnostics System.Diagnostics.FileVersionInfo different/incorrect behaviors compared to .NET Framework FileVersionInfo solely depends on System.IO.FileSystem (and all of its dependencies, including System.IO, System.Threading.Tasks, etc) just to do a File.Exists in GetVersionInfo. We should remove this and instead see if we can use the return value of GetFileVersionInfoSizeEx to mimic the same behavior. 283 area-System.Linq System.Linq.Parallel test assembly disabled as it takes too long on AppVeyor The System.Linq.Parallel test assembly is disabled because it takes 50 seconds to run on my development machine and lots and lots of time to run on AppVeyor. 284 area-System.Xml XPath AxesCombination tests are failing on debug builds While running with debug builds the following AxesCombination Tests are failing with the below assert: \src\Common\tests\System.Xml.XPath\FuncLocation\PathAxeCombinationsTests.cs AxesCombinationsTest2142 AxesCombinationsTest2143 AxesCombinationsTest2144 IsSamePosition() on custom navigator returns inconsistent results at XPathNavigator.ComparePosition(XPathNavigator nav) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\XPathNavigator.cs(711) at Query.CompareNodes(XPathNavigator l, XPathNavigator r) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\Internal\Query.cs(150) at Query.Insert(List`1 buffer, XPathNavigator nav) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\Internal\Query.cs(109) at DocumentOrderQuery.Evaluate(XPathNodeIterator context) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\Internal\DocumentOrderQuery.cs(20) at BaseAxisQuery.Evaluate(XPathNodeIterator nodeIterator) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\Internal\BaseAxisQuery.cs(108) at XPathNavigator.Evaluate(XPathExpression expr, XPathNodeIterator context) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\XPathNavigator.cs(802) at XPathNavigator.Evaluate(XPathExpression expr) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\XPathNavigator.cs(784) at XPathNavigator.Select(XPathExpression expr) d:\github\corefx\src\System.Xml.XPath\src\System\Xml\XPath\XPathNavigator.cs(764) at Utils.XPathNodesetTest(String xml, String testExpression, XPathResult expected, XmlNamespaceManager namespaceManager, String startingNodePath) d:\github\corefx\src\Common\tests\System.Xml.XPath\Common\Utils.cs(128) at AxesCombinationsTests.AxesCombinationsTest2144() d:\github\corefx\src\Common\tests\System.Xml.XPath\FuncLocation\PathAxeCombinationsTests.cs(44979) 292 area-System.Text Add Regex Tests This adds our tests for the System.Text.RegularExpressions library. Currently, these tests (and almost all of the tests that we will be bringing out from our internal sources) are going to print a lot of things to the console, as that is the testing paradigm we follow internally. We will be coming up with a way to either redirect the test output, or make it configurable. 293 area-System.Diagnostics Add FileVersionInfo tests This adds all of our tests for the System.Diagnostics.FileVersionInfo library. Included also are the assemblies and executables that are used as test assets. 294 area-Meta Proposal for the API Review process Now that we’re on GitHub we also get requests for new APIs. The current API review process is designed around the assumption that it is internal-only. We need to rethink this process for an open source world. This issue represents a proposal how this could be handled. It’s not final – any feedback is highly appreciated. ## Process Goals The key goals are: - **Designed for GitHub**. In order to be sustainable and not be a hurdle for contributors the API review process must feel natural to folks familiar with GitHub. - **Efficiency**. Performing API reviews requires looping in a set of experts. We want to conduct API reviews in an agile fashion without randomizing the reviewers or community members. - **Transparency**. We can use the same process for both internal as well as external contributors. This allows contributors to benefit from the results of API reviews even if the implementer isn’t external. ## Overall Process GitHub is generally based around the pull-request model. The idea is that contributors perform their changes in their own fork and submit a pull request against our repository. For trivial code changes, such as typo fixes, we want folks to directly submit a pull request rather than opening an issue. However, for bug fixes or feature work, we want contributors to first start a discussion by creating an issue. For work that involves adding new APIs we'd like the issue to contain what we call a _speclet_. The speclet should provide a rough sketch of how the APIs are intended to be used, with sample code that shows typical scenarios. The goal isn't to be complete but rather to illustrate the direction so that readers can judge whether the proposal is sound. ![Process diagram](https://cloud.githubusercontent.com/assets/5169960/5499091/a6895e2c-86dd-11e4-8b0e-6ba1ae152541.png) ## Steps - **Contributor opens an issue**. The issue description should contain a speclet that represents a sketch of the new APIs, including samples on how the APIs are being used. The goal isn’t to get a complete API list, but a good handle on how the new APIs would roughly look like and in what scenarios they are being used. - **Community discusses the proposal**. If changes are necessary, the contributor is encouraged to edit the issue description. This allows folks joining later to understand the most recent proposal. To avoid confusion, the contributor should maintain a tiny change log, like a bolded “Updates:” followed by a bullet point list of the updates that were being made. - **Issue is tagged as “Accepting PRs”**. Once the contributor and project owner agree on the overall shape and direction, the project owner tags the issue as “Accepting PRs”. The contributor should indicate whether they will be providing the PR or only contributed the idea. - **Coding**. The contributor is implementing the APIs as discussed. Minor deviations are OK, but if during the implementation the design starts to take a major shift, the contributor is encouraged to go back to the issue and raise the concerns with the current proposal. - **Pull request is being created**. Once the contributor believes the implementation is ready for review, she creates a pull request, referencing the issue created in the first step. - **Pull request is being reviewed**. The community reviews the code for the pull request. The review should focus on the code changes and architecture – not the APIs themselves. Once at least two project owners give their OK, the PR is considered good to go. - **Pull is tagged as “Needs API Review”**. The project owner then marks the pull request as “Needs API Review”. - **API review**. Using the information in the pull request we’ll create an APIX file that constitutes the API delta. The API review board meets multiple times a week to review all PRs that are tagged as needing an API review. - **Pull request is updated with the results of the API Review**. Once the API review is complete, the project owner uploads the notes and API HTML diff, including all comments. The project owner also updates the PR accordingly, with either a call to action to address some concerns or a good to go indicator. - **Pull request is merged**. When there are no issues – or the issues were addressed by the contributor, the PR is merged. ## API Design Guidelines The .NET design guidelines are captured in the famous book [Framework Design Guidelines](http://amazon.com/dp/0321545613) by Krzysztof Cwalina and Brad Abrams. A digest with the most important guidelines are available in our [developer wiki](https://github.com/dotnet/corefx/wiki/Framework%20Design%20Guidelines%20Digest). Long term, we'd like to publish the individual guidelines in standalone repo on which we can also accept PRs and -- more importantly for API reviews -- link to. 295 area-System.Diagnostics Fix file name casing in System.Diagnostics.FileVersionInfo.csproj The file on disk is called Interop.manual.cs, not Interop.Manual.cs. This fixes compilation with Mono on Linux (or other case-sensitive FS). 298 area-System.Xml Use String.Equals instead of String.Compare for equality checks Per the [Best Practices for Using Strings in the .NET Framework](http://msdn.microsoft.com/en-us/library/dd465121%28v=vs.110%29.aspx): - Use an overload of the `String.Equals` method to test whether two strings are equal. - Use the `String.Compare` and `String.CompareTo` methods to sort strings, **not to check for equality**. 299 area-Meta Removing `readonly` from a field can be a breaking change Currently the [**Signatures** section of the Breaking Change Rules](https://github.com/dotnet/corefx/wiki/Breaking-Change-Rules#signatures) page includes the following: > ✓ Allowed > - Removing `readonly` from a field As hinted in [a recent tweet](https://twitter.com/samharwell/status/543970816321650688), this should probably be disallowed in a particular case. I would prefer not to give the answer away publicly yet, but feel free to contact me by email. Here is the original tweet: > C# Quiz: Add `readonly` to a field only set in the constructor, which changes the runtime behavior of the type (no use of reflection). 300 area-Meta Refactor System.Servicemodel.ClientBase to have non-generic base "The following is an example of a generic extension method I would like to write, but cannot because `ClientBase` does not inherit from a base class containing the non generic members. It would be very helpful if `ClientBase` was refactored into `ClientBase:ClientBase` ``` C# using DotNetOpenAuth.OAuth2; using System; using System.Net; using System.ServiceModel; using System.ServiceModel.Channels; using System.Threading; using System.Threading.Tasks; namespace OAuthClient { public static class ExtensionMethods { public static async Task CallAsync(this TClient wcfClient, Func predicate, IAuthorizationState authorization, CancellationToken cancellationToken) where TClient : ClientBase where TChannel : class { if (authorization == null) { throw new InvalidOperationException(""No access token!""); } // Refresh the access token if it expires and if its lifetime is too short to be of use. if (authorization.AccessTokenExpirationUtc.HasValue) { await AuthorizationServer.Client.RefreshAuthorizationAsync(authorization, TimeSpan.FromSeconds(30)); } var httpRequest = (HttpWebRequest)WebRequest.Create(wcfClient.Endpoint.Address.Uri); ClientBase.AuthorizeRequest(httpRequest, authorization.AccessToken); var httpDetails = new HttpRequestMessageProperty(); httpDetails.Headers[HttpRequestHeader.Authorization] = httpRequest.Headers[HttpRequestHeader.Authorization]; using (var scope = new OperationContextScope(wcfClient.InnerChannel)) { OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = httpDetails; bool isError = true; try { T result = predicate(wcfClient); // If there is an error on the channel, the close call will throw an exception. wcfClient.Close(); isError = false; return result; } finally { // If we have an error on the channel, we cannot close the channel so we abort. // Exceptions that occur will still be raised unmodified. if (isError) { wcfClient.Abort(); } } } } } } ``` " 301 area-System.Diagnostics FileVersionInfo Cleanup This addresses code review feedback from #274. /cc @mellinoe and @stephentoub 304 area-System.Text Regex should provide a validation method "`Regex` doesn't provide a way to validate whether a given string is a valid regular expression without getting an exception. It should provide one. ## Background `TryParse` is a common pattern in .NET which is useful when a potentially invalid string has to be converted to a different representation. As invalid user input is not an exceptional situation, it makes sense to avoid exception overhead (which coincidentally also educates new developers to handle those situations correctly). Regular expressions are not as common in external input as `int` or `DateTime`, however there are situations when regex is an appropriate tool for the power users (e.g. advanced text editor search/replace). If you want to validate the regex as it is being typed (with reasonable throttling), it is useful to have as little overhead as possible. Request for Regex.TryParse (not by me) had [5 votes on Microsoft Connect](http://webcache.googleusercontent.com/search?q=cache:KV6wcz2SJ44J:https://connect.microsoft.com/VisualStudio/feedback/details/331753/regex-should-have-a-static-tryparse-method+) and currently exists on [UserVoice with 3 votes](http://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/2683916-regex-should-have-a-static-tryparse-method). ## Design Options Originally I wanted to propose `Regex.TryCreate` (which is in line with `Uri.TryCreate` since `Regex` does no have a `Parse` method). However even for power users regexes might not be easy to write, and providing more information than just ""you have failed"" is useful. I can't think of any existing pattern in the BCL that fits this well, but basically something like this would be useful: ``` csharp public class Regex { // existing members… public static RegexParseResult Parse(string pattern); public static RegexParseResult Parse(string pattern, RegexOptions options); // or alternatively we could use a default value for options } public class RegexParseResult { public bool IsSuccess { get; } public Regex Regex { get; } // parser doesn't have to provide more than one, however it would allow it to do so in the future public IReadOnlyCollection Errors { get; } } public class RegexParseError { public string Message { get; } public int Start { get; } public int End { get; } } ``` ### API Concerns It would be useful to know if I am missing some existing common pattern here, so that consistency with other BCL APIs can be provided. I don't like using name `Parse` here either as it invites confusion -- however I can't currently think of a better approach. ## Example Usage ``` csharp public class RegexValidationRule : System.Windows.Controls.ValidationRule { public override ValidationResult Validate(object value, CultureInfo cultureInfo) { var valueString = (string)value; var parseResult = Regex.Parse(valueString); if (!parseResult.IsSuccess) { var errorString = string.Join(NewLine, parseResult.Errors.Select(e => $""{e.Message} at {e.Start}"")); return new ValidationResult(false, errorString); } return ValidationResult.ValidResult; } } ``` ## Compatibility I think adding a new static method should be safe enough. " 306 area-System.Diagnostics Make Process.Start have a option to change handle inheritance "Currently if you call Process.Start internally CreateProcess is called with bInheritHandles = true (hard coded). It would be great to make it possible to change this behavior, e.g. by adding a Property to ProcessStartInfo. Currently there is no way I know of to change this other then reimplementing System.Diagnostics.Process. # Example If you run this application twice without exiting the first notepad instance the second instance will not be able to open the tcp port, because notepad is still running. This can be a problem for server applications that are starting child processes themself and crash, or are killed by the user before the socket can be closed. ``` csharp using System.Diagnostics; using System.Net; using System.Net.Sockets; class Program { static void Main() { TcpListener listener = new TcpListener(IPAddress.Any, 4567); listener.Start(); Process.Start(new ProcessStartInfo(""notepad.exe"") { UseShellExecute = false }); //Simulate application crash without freeing resources } } ``` # Design proposal The easiest way to make this possible is to add a new Property to ProcessStartInfo and use this in the Call to CreateProcess ``` csharp public sealed class ProcessStartInfo { // ... public bool InheritHandles { get; set; } // defaults to true // ... } ``` # Questions - Is there a very important reason why this was hardcoded like this in the first place? " 307 area-Meta Is the corefx repo or the dotnet org a good place for key community projects? Forked from #294. from @n8ohu - > I am working on uses a C# wrapper around a cross-platform C library and it would, in my opinion, be beneficial to other developers if it was part of the core; There is really a spectrum of .NET libraries, with one end point being the official corefx distribution and the other a weekend project never to be touched again. What you are proposing is much closer to the former. I think your base question is whether corefx or a peer repo is appropriate for key community-owned projects. Your scenario is C# wrappers over native libraries, but it could also be a pure managed library, too. We are considering a similar plan for our own libraries that don't fit or don't fit yet in corefx. We can and should include community libraries in that thinking. The Mono org seems to embrace community libraries that have broad applicability. We should take some inspiration from that. It's always a fine idea to host a library out of your own GitHub account. At the same time, we are intending the dotnet org to be a center of gravity for .NET library development. That's why we put the corefx repo there (and same will be true for the coreclr one) instead of the Microsoft org. It's likely the case that discoverability will be better in the dotnet org than an arbitrary personal repo. What do you think? 311 area-System.Console Bring back Console.CancelKeyPress "We need this so that we can wait on Ctrl + C events in console applications. See https://github.com/aspnet/Hosting/blob/dev/src/Microsoft.AspNet.Hosting/Program.cs#L69. " 312 area-System.IO FileSystem cross platformness and testability Building cross platform applications that talk to the file system is painful. There are several issues like case sensitivity, slash differences and other bizarre restrictions and differences. On another front, testability of the file system has always been impossible with .NET. Any application/library that uses the FileSystem uses something like https://www.nuget.org/packages/System.IO.Abstractions or a custom abstraction to allow mockability. Are there any plans to improve the System.IO API to solve some of these problems? 313 area-System.Numerics System.Numerics.Vectors Fails to Load W/ .NET 4.6 Preview Installed "``` An unhandled exception of type 'System.IO.FileLoadException' occurred in Unknown Module. Additional information: Could not load file or assembly 'System.Numerics.Vectors, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) ``` Steps to Reproduce. - Create a console application. - Add the nuget package for System.Numerics.Vector to the project (I tried 1.1.5-beta and 1.1.6-beta). - Add some dependent code (I used `Console.WriteLine(""Hardware Accelerated Vectors: {0}"", Vector.IsHardwareAccelerated);`) - Run the project. - Get above exception. It looks like somehow VS is getting the version number pinned at 4.0.0 and rejects the load. Specific version is disabled, the DLL is being copied as expected and no relevant entries appear to be in the app.config. " 316 area-System.Text Implement IList, IReadOnlyList, and IList on Regex Collections Fixes #271 This is still a work-in-progress. ### TODO - [x] Implement `IList`, `IReadOnlyList`, and `IList` - [x] Add `RegexCollectionDebuggerProxy` - [x] API review - [x] Changes based on API review and initial code review feedback - [ ] Cleanup - [ ] Add tests - [ ] Final code review ### Questions/Notes - As @sharwell noted in #277: `GroupEnumerator` is internal, so it might be possible to change the type of `GroupEnumerator.Capture` from `Capture` to `Group`. This would make the explicit cast in the generic `Current` property unnecessary. 317 area-System.Text Simplify MatchCollection's ICollection.CopyTo - Uses [`List`'s implementation of `ICollection.CopyTo`](https://github.com/Microsoft/referencesource/blob/9da503f9ef21e8d1f2905c78d4e3e5cbb3d6f85a/mscorlib/system/collections/generic/list.cs#L394-L407) which is behavior compatible with the previous implementation. - Avoids an unnecessary array allocation by no longer using `List.ToArray()`. - `SR.Arg_RankMultiDimNotSupported` and `SR.Arg_InvalidArrayType` are no longer needed. - Avoids accessing a property to trigger a side effect. 321 area-System.Text Regex capture, match and group collections now implement the generic IEnumerable This is my first contribution to the .NET Core. I would like to know whether this change would be acceptable under the guidelines of the .NET framework. I am asking because I am making classes implement an interface (acceptable by your rules), but I am also changing the existing interface implementation to an explicit one, in favor of the implicit implementation being the new interface (which is a really grey area that is not included in the rules). Please guide me as to how this pull request needs to change to become acceptable. 322 area-Meta Branching (or not) While commenting on [an outstanding pull request](https://github.com/dotnet/corefx/pull/110#issuecomment-65910391), David Kean mentioned: > We're trying to figure out our branching structure at the moment. Currently we have only a single branch that represents what we're shipping for the next update of .NET Core. Clearly this isn't scalable for the product, and we're going to creating branches for future work so PR's that have API changes and destablizing changes get a little more bake time. @joshfree is looking into that. I would like to get the conversation about branching structure out into the open. And to do that, I would like to start by asking: _why_ is a single branch not scalable for the product? I have worked in code bases of various sizes, release schedules, and branching schemes, and my experience is that NOT branching is nearly always preferable to branching. In fact, the most common reasons given for wanting to branch (e.g. disruptive or long-running changes) are exactly the times when branching hurts the most, because it encourages divergence between different works-in-progress, pushing difficult integration decisions farther out and requiring more rework later. Rather than a policy that handles change by branching, I would rather see a policy of trunk-based development that encourages all changes - even incomplete changes - to be merged into master on a frequent basis. This shortens and simplifies the merging process and encourages small refactorings and improvements by reducing the amount of work needed to merge them. Obviously allowing unfinished work to be committed to master requires mechanisms in place to ensure that such work does not affect the releasability of the framework. Mechanisms such as feature flags and branch by abstraction are slightly more complicated to implement in a framework (especially one that is highly tuned for performance) than in a typical application, but not excessively so. And the advantage of keeping all of the ongoing work together is well worth the effort. What are the community's thoughts on trunk-based development? 323 area-System.Text Decrease in working set was incorrectly marked as test failure "I am hitting a test failure that does not have a consistent repro. ``` Err_004888anied Total memory increased significantly there may be a memory leak (TaskId:339) Percent Difference=0.338227637196709, Max Acceptable Percent Difference=0.3, Start Working Set=576180, End Working Set=381300 (TaskId:339) ``` We probably should not use `Math.abs()` to calculate the difference at https://github.com/dotnet/corefx/blob/master/src/System.Text.RegularExpressions/tests/RegexLangElementsCoverageTests.cs#L60 ``` csharp percentDifference = ((double)Math.Abs(startWorkingSet - endWorkingSet)) / startWorkingSet; if (maxAcceptablePercentDifference < percentDifference) { Console.WriteLine(""Err_004888anied Total memory increased significantly there may be a memory leak""); Console.WriteLine(""Percent Difference={0}, Max Acceptable Percent Difference={1}, Start Working Set={2}, End Working Set={3}"", percentDifference, maxAcceptablePercentDifference, startWorkingSet, endWorkingSet); iCountErrors++; } ``` " 325 area-System.Numerics StringBuilder append with single length strings changed to chars Some StringBuilder.Append calls made with single length symbol strings. These can be a char so that they get directly assigned to the internal char array buffer. Also more consistent with other areas of the codebase. 326 area-Meta .NET Core and pattern compatability (@terrajobst rightly [pointed out](https://github.com/dotnet/corefx/issues/271#issuecomment-68022859) that the thread on implementing collection interfaces on Regex classes had wandered into abstract discussion not related to the original issue. So I am opening this issue in order to continue the discussion here.) To summarize my point (see the [Regex thread](https://github.com/dotnet/corefx/issues/271) for more details): with the addition of IReadOnlyList in .NET 4.5, I don't believe it is appropriate to continue implementing mutable collection interfaces such as IList on immutable collections. The fact that existing immutable collections (including those in the Immutable Collections library) implement these interfaces is not a good enough reason to continue this flawed pattern. This issue is a particular example of the larger question of how .NET Core will evolve, and how much tolerance it has to evolve _away_ from flawed patterns in the .NET Framework. 329 area-Infrastructure Use `nuget restore` instead of `nuget install` See [this line](https://github.com/dotnet/corefx/pull/324#discussion-diff-22273168R41) and the discussion for details. 338 area-System.Text RegexCharClass Improvements and Cleanup Apologies in advance for the large pull request. Normally, I'd break these changes up into separate smaller pull requests, but in this case the changes were all to the same file and some changes build on top of earlier changes, so it was easier to submit as one larger pull request. Each commit is factored appropriately, so it may be easiest to review each commit separately for more sane diffs. The most significant change was to precompute the constants instead of computing the values at runtime in the static constructor. I have a [one-off test](https://gist.github.com/justinvp/0c1b5faf72349b56a2ed) (not intended to be checked-in) that asserts that the precomputed values from the new implementation are equal to the values that would have been computed at runtime using the previous implementation. 340 area-System.Text Support for RegexOptions.Compiled? `RegexOptions.Compiled` isn't currently supported (the option exists but doesn't do anything). Will this be supported? If so, when? If not, why not? 342 area-System.Text More Regex Cleanup Sorry for another larger pull request -- there are just so many opportunities for cleanup in Regex! Most of the commits are pretty fine-grained (some probably could have been squashed together, sorry). 344 area-System.Security Support for Rfc2898DeriveBytes and other founding cryptographic support Hi, Reading the contribution DOs and DON'TS it says not to file an API addition without discussing first. I have the System.Security.Cryptography.Rfc2898DeriveBytes essentially created in a PCL so works in WinRT etc. I'd like to expand on this as Rfc2898DeriveBytes is HmacSha1 based and people are trying to move onto hmacsha256 and upwards nowadays and I'd like to bring this into .NET Core I'd like to start bringing in System.Security.Cryptography namespace into .NET Core. I'll include tests from the test data listed with the cryptographic standards and whatnot where possible. Can I do this? :) 348 area-Meta Updated README.md with recently added projects 349 area-System.Threading TaskHelpers cleanup TaskHelpers.cs is a temporary stop-gap for assemblies that don't yet depend on the latest System.Threading.Tasks contract. This change removes the dependency on the file for those assemblies that do use the latest, changing them to access the Task.CompletedTask, Task.FromCanceled, and Task.FromException members on Task instead of the corresponding workarounds on TaskHelpers. This file should eventually go away entirely. 350 area-System.Console Use StringBuilderCache in additional assemblies "StringBuilderCache maintains a cache of up to a single StringBuilder per thread, avoiding StringBuilder allocations for the common pattern of: ``` C# StringBuilder sb = new StringBuilder(); // ... do some appends return sb.ToString(); ``` and instead enabling: ``` C# StringBuilder sb = StringBuilderCache.Acquire(); // ... do some appends return StringBuilderCache.GetStringAndRelease(sb); ``` It was added to the repo as part of System.IO.FileSystem. This change deploys usage of it through other assemblies. Not all ""new StringBuilder(...)"" usage has been replaced, just those instances that are expected to be most impactful, e.g. tests were ignored, places where StringBuilders are stored into fields were ignored, debug code was ignored, less common members were ignored, etc. Simple microbenchmarks showed measurable improvements as a result. For example, a test around Regex.Replace showed that ~10% of the allocations incurred were for StringBuilder instances; using StringBuilderCache mostly eliminated these and improved throughput by a corresponding ~10%. This will of course vary based on the test employed, but it's a data point. " 352 area-Meta Design document for performance patterns As noted in #351, some patterns which are commonly used in the .NET Framework for performance reasons are not obvious to users who are new to this project. Due to the number of projects which leverage the .NET library as part of their own applications, performance is a particularly sensitive area, especially in regards to [unnecessary] memory allocations. It would be very helpful to have a document explaining commonly used patterns intended to avoid unnecessary memory allocations, and perhaps other techniques which your prior work has indicated are high-value practices for some reason. 361 area-System.Net Ability to set the HttpWebRequest.DefaultMaximumErrorResponseLength on a per request basis I realize the http code hasn't been imported yet.. But it would be really nice if this could be changed in >=4.6 Currently the only way you can set the DefaultMaximumErrorResponseLength for a request is AppDomain wide. We should also be able to be set on a per request basis (like we can do with ServicePoint). 362 area-System.Console Bring back Console.CancelKeyPress We need this so that we can wait on Ctrl + C events in console applications. issue #311 371 area-System.IO Unix Interop Code is platform specific In commits like this: https://github.com/dotnet/corefx/commit/377dde76c5df105a7bab2c0a3002c0d835ff1b07 An attempt has been made to map both structures and constants to C# managed structures. Unix does not have a binary interface for either the constants or the structures, so the definition is likely already wrong for some platforms and will likely be wrong for ports, even within Linux itself. While the structures are constants do change from Unix to Unix a little known quirk is that both structures and constants also change in size, layout and the values in Linux itself. Early versions of Linux chose to emulate the ABI of the host platform, allowing early Linux to easily execute binaries written for the mainstream operating system of the platform. Linux/Alpha copied OSF/1; Linux/SPARC copied SunOS 4; Linux/MIPS some variation of IRIX and so on. This is why Linux includes many of these definitions in /usr/include/arch/XXX, because they are different in each platform. You should do something similar to what Mono.Posix does, which is to have a managed wrapper that talks to a C library that actually performs the native call and marshals the native version to a well-known structure shared with the managed implementation. 377 area-System.Text Regex tests intermittently fail due to memory usage "In RegexLangElementsCoverageTests.cs, there are several checks related to the total memory usage of the test: ``` CSharp // ** RegexLangElementsCoverageTests.cs ** double maxAcceptablePercentDifference = .3; startWorkingSet = GC.GetTotalMemory(true); ... endWorkingSet = GC.GetTotalMemory(true); percentDifference = ((double)Math.Abs(startWorkingSet - endWorkingSet)) / startWorkingSet; if (maxAcceptablePercentDifference < percentDifference) { Console.WriteLine(""Err_004888anied Total memory increased significantly there may be a memory leak""); Console.WriteLine(""Percent Difference={0}, Max Acceptable Percent Difference={1}, Start Working Set={2}, End Working Set={3}"", percentDifference, maxAcceptablePercentDifference, startWorkingSet, endWorkingSet); iCountErrors++; } ``` I think this check may be causing us more bad than good. I'm not sure what GC.GetTotalMemory takes into consideration, but this logic might also be getting tripped up by the fact that we have multiple tests running in parallel. Also, in general, this seems like a bit of a flaky check in the first place; it fails around 10% of the time for me, I would say. Anyone opposed to removing these checks, or at least making it not an error? " 381 area-Meta Question: Single statement `if` formatting "Looking at `System.Threading.Tasks.Dataflow` (amazing library by the way), I see three different variations of formatting of single-statement `if` statements. In _one_ method. From [`DataflowBlock.OutputAvailableAsync`](https://github.com/dotnet/corefx/blob/2bf5a7185fd81949da4d47a432662d8fe80989f6/src/System.Threading.Tasks.Dataflow/src/Base/DataflowBlock.cs#L1485-L1550) [**One line**](https://github.com/dotnet/corefx/blob/2bf5a7185fd81949da4d47a432662d8fe80989f6/src/System.Threading.Tasks.Dataflow/src/Base/DataflowBlock.cs#L1489): ``` c# if (source == null) throw new ArgumentNullException(""source""); ``` [**Two lines**](https://github.com/dotnet/corefx/blob/2bf5a7185fd81949da4d47a432662d8fe80989f6/src/System.Threading.Tasks.Dataflow/src/Base/DataflowBlock.cs#L1493-L1494): ``` c# if (cancellationToken.IsCancellationRequested) return Common.CreateTaskFromCancellation(cancellationToken); ``` [**With braces**](https://github.com/dotnet/corefx/blob/2bf5a7185fd81949da4d47a432662d8fe80989f6/src/System.Threading.Tasks.Dataflow/src/Base/DataflowBlock.cs#L1513-L1516): ``` c# if (target.Task.IsCompleted) { return target.Task; } ``` Are the guidelines for formatting these more nuanced than I'm able to see, or is this just inconsistent formatting? Is this kind of inconsistent formatting normal and acceptable? Is there a preference for how new code should be formatted? " 384 area-System.Collections Application of ObsoleteAttribute to legacy collection types As mentioned in #370, the lack of `[Obsolete]` on certain legacy types in `System.Collections.Specialized` could be confusing to developers. In general I support the use of ObsoleteAttribute, but for this particular library I would recommend it only be applied to types which have well-known newer (better) alternatives. I'm working on the following table... | Legacy Type | Replacement | | --- | --- | | `BitVector32` | No direct replacement? | | `HybridDictionary` | No direct replacement? | | `IOrderedDictionary` | No direct replacement? | | `ListDictionary` | No direct replacement? | | `NameObjectCollectionBase` | No direct replacement? | | `NameValueCollection` | No direct replacement? | | `OrderedDictionary` | No direct replacement? | | `StringCollection` | `List` | | `StringEnumerator` | Only used with `StringCollection` (not marked obsolete?) | | `StringDictionary` | `Dictionary` | 386 area-Infrastructure Jenkins connection abort issues and associated issues with subsequent runs Seeing intermittent issues with Windows remoting in Jenkins. These are known issues but have gone unfixed for a while. In addition, this causes the next job to run in a potentially un-clean environment 391 area-Meta Language or BCL data types "I'm not sure it's worth to discuss this issue. To ensure consistency within the code base the preferred data types should be noted in the wiki. I know this can be a sensitive and subjective issue... **Only use BCL data types** ``` c# String text; String.IsNullOrEmpty(text); Double number; Double.TryParse(text, out number); ``` **Only use language data types** ``` c# string text; string.IsNullOrEmpty(text); double number; double.TryParse(text, out number); ``` **Use language data types for declarations | BCL data types for calling members** ``` c# string text; String.IsNullOrEmpty(text); double number; Double.TryParse(text, out number); ``` My favorit is the last option because - an Enum can't inherit from UInt32 so you have to use uint - because it's not obvious that data types have members - ~~[MSDN](http://msdn.microsoft.com/en-us/library/ms229045.aspx) says ""**√ DO** use a generic CLR type name, rather than a language-specific name, in the rare cases when an identifier has no semantic meaning beyond its type.""~~ " 392 area-Meta Update Wiki to mention that changes to .NET Core must be binary compatible with the full .NET Framework A question came up in the comments for #271 about the possibility of a binary breaking change in the .NET Core Regex collections from the Regex collections in the full .NET Framework. @davkean responded: > You need to be able to build against this library and run on .NET Framework 4.6. And I asked: > So if you accept this suggestion (adding the generic interfaces to these types), the changes will have to be added to .NET 4.6 as well? How much time is left to make changes to .NET 4.6? And @weshaggard responded: > Yes @justinvp any types that already exist on the full desktop framework will need to be updated there as well if they are updated in .NET Core. We have a high level goal of making code written for .NET work no matter what .NET platform you are targeting. As for how much time is left to add new public APIs to .NET 4.6 at this point we are not accepting any more unless they are critical. > > With that said we fully expect .NET Core to evolve much more quickly then the full .NET framework that ships in Windows. We just have to be aware that any changes need to eventually make it there as well so while thinking about breaking changes you cannot just consider .NET Core you must consider the other platforms as well. If we don't do that then we risk diverging the platforms and making .NET development as a whole more difficult. I think it'd be worth adding this to one of the wiki pages so it is clear to everyone that types in .NET Core that exist in the full .NET Framework must be binary compatible. I don't think this is explicitly stated anywhere currently. 395 area-System.Runtime String comparer for sorting numeric strings logically "## Rationale For sorting purposes it's common to need portions of strings containing numbers to be treated like numbers. Consider the list of strings `""Windows 7"", ""Windows 10""`. Using the `Ordinal` `StringComparer` to sort the list one would get ``` Windows 10 Windows 7 ``` but the desired ascending logical sort would be ``` Windows 7 Windows 10 ``` ## Proposed API ```diff namespace System { public class StringComparer { + public static StringComparer Create(CultureInfo culture, CompareOptions options); } } namespace System.Globalization { public enum CompareOptions { + NumericOrdering = 0x00000020 } } ``` ## Usage ```cs var list = new List { ""Windows 10"", ""Windows 7"" }; list.Sort(StringComparer.Logical); // List is now ""Windows 7"", ""Windows 10"" ``` This would also be good for sorting strings containing IP addresses. ## Details * `Logical` is a convenience property equivalent to the result of `Create(CultureInfo.CurrentCulture, CompareOptions.Logical)` * `LogicalIgnoreCase` is a convenience property equivalent to the result of `Create(CultureInfo.CurrentCulture, CompareOptions.Logical | CompareOptions.IgnoreCase)` * Non-numeric sequences will be evaluated with the culture provided. * Numeric sequences will be determined by the result of `Char.IsDigit`. * All UTF-16 digits will be supported and are manually parsed using `Char.GetNumericValue`. * Only positive integral values without digit separators will be supported directly. * Numbers will be treated as `ulong`s. Logic for overflows will have to be considered. * The string `Windows 8.1` would be considered 4 sequences. The `Windows ` would be a string sequence, the `8` would be a numeric sequence, the `.` would be another string sequence, and the `1` would be another numeric sequence. * This API could later be expanded to include support for allowing signs, decimals, and digit separators through the use of overloads accepting a `NumberStyles` parameter. * When a numeric and string sequence are considered at the same time the numeric sequence always comes before the string sequence so when sorting the following list, `""a"", ""7""` the number `7` will be sorted before the letter `a`. * Existing methods that take a `CompareOptions` parameter as input will need to be updated to support the new `Logical` member. ## Open Questions * Should `CompareOptions.Logical` be implemented as the flag option `SORT_DIGITSASNUMBERS` to the `dwCmpFlags` parameter of [`CompareStringEx`](https://msdn.microsoft.com/en-us/library/windows/desktop/dd317761(v=vs.85).aspx)? Using it's implementation should be more efficient but later expanding support for `NumberStyles` will require a re-implementation with matching behavior. ## Updates * Added `Logical` and `LogicalIgnoreCase` properties. * Added support for all UTF-16 digits. * Added more `CreateLogical` overloads to match the `Create` method. * Added retrieval of the `NumberFormatInfo` from the `StringComparer` parameter when not explicitly provided and is a `CultureAwareComparer`. * Removed `CreateLogical` overloads that matched the `Create` method. * Switched to only supporting positive integral values without digit separators. * Added consideration of comparing a numeric sequence with a string sequence. * Added the flag member `CompareOptions.Logical` and changed `CreateLogical` to be just an overload of `Create`." 396 area-Meta Update README.md description of legacy collections 398 area-System.Diagnostics Refactor System.Diagnostics.FileVersionInfo with PAL Introduce a Windows and Unix build for System.Diagnostics.FileVersionInfo.dll. Windows-specific functionality has been moved to separate files, with Unix-specific (but stubbed out) files added to match. 409 area-System.Xml Reduce XPath unit tests run time "Either reduce unit tests subset (apply [OuterLoop] to long tests) or modify tests to keep run time below 5s on average machine. Currently on some machine they run for ~25s. Reduce it by a factor of 5. " 410 area-Meta Update readme to reflect System.Collections.Concurrent 423 area-Meta CLA Labels I've been following the project for a while now and noticed that recently there is a bot in place that adds `cla-required`, `cla-signed` and `cla-not-required`. I personally think that these labels clutter up a lot. Wouldn't it be easier to only add `cla-required` when they haven't signed it yet and remove it when signed? Leaving out the other two labels. Also, when signing up for the CLA you require to enter the company you work for, not everyone has work and some are still studying. 424 area-System.Security X509Certificate: .pfx format unsupported on CoreCLR "This snippet works on `aspnet50` but an exception is thrown on `aspnetcore50` (tested on Windows 7 and Windows 8.1): ``` csharp public static class Program { public static void Main(string[] args) { var assembly = typeof(Program).GetTypeInfo().Assembly; using (var resource = assembly.GetManifestResourceStream(""Certificate.pfx"")) using (var buffer = new MemoryStream()) { resource.CopyTo(buffer); var certificate = new X509Certificate2(buffer.ToArray(), ""Owin.Security.OpenIdConnect.Server""); Console.ReadLine(); } } } ``` ![image](https://cloud.githubusercontent.com/assets/6998306/5743652/fdeb3562-9c1a-11e4-93db-1fec2523adcd.png) ``` {System.Security.Cryptography.CryptographicException: Impossible de trouver l’objet requis. at System.Security.Cryptography.CryptographicException.ThrowCryptographicException(Int32 hr) at System.Security.Cryptography.X509Certificates.X509Utils._QueryCertBlobType(Byte[] rawData) at System.Security.Cryptography.X509Certificates.X509Certificate.LoadCertificateFromBlob(Byte[] rawData, Object password, X509KeyStorageFlags keyStorageFlags) at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password) at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password) at X509Certificate2.Program.Main(String[] args)} ``` Complete repro: https://github.com/PinpointTownes/X509Certificate-Bug " 432 area-System.Console Fix handling of some terminfo files ConsolePal in Unix parses terminfo files to determine the correct format strings to use for setting foreground and background color. Before indexing into the file's contents to grab those strings, we were missing a check to ensure that the file actually contained enough string entries that it could contain a foreground or background color string. This commit just adds that check. 439 area-Infrastructure Building in Visual Studio stopped working While building on the command line via `build.cmd` works just fine, building with Visual Studio doesn't. It's worth calling out that building on the command line via `devenv` is also broken: ``` $ devenv .\System.Collections.NonGeneric.sln /build Microsoft Visual Studio 2013 Version 12.0.31101.0. Copyright (C) Microsoft Corp. All rights reserved. 1>------ Build started: Project: System.Collections.NonGeneric, Configuration: Debug Any CPU ------ 1>CSC : error CS0518: Predefined type 'System.Object' is not defined or imported 2>------ Build started: Project: System.Collections.NonGeneric.Tests, Configuration: Debug Any CPU ------ 2>CSC : error CS0006: Metadata file 'P:\oss\corefx\bin\Debug\System.Collections.NonGeneric\System.Collections.NonGeneric.dll' could not be found ========== Build: 0 succeeded, 2 failed, 0 up-to-date, 0 skipped ========== ``` It seems the new build-based package resolution is to blame. For example, here is the output of building **System.Collections.NonGeneric**: | Type | # | Description | Project | | --- | --- | --- | --- | | Error | 1 | Unable to find nupkg for System.Text.Encoding. | System.Collections.NonGeneric.Tests | | Error | 2 | Unable to find nupkg for xunit.console.netcore. | System.Collections.NonGeneric.Tests | | Error | 3 | Unable to find nupkg for xunit.runner.dependencies.netcore. | System.Collections.NonGeneric.Tests | | Error | 4 | Unable to find nupkg for Microsoft.DotNet.TestHost. | System.Collections.NonGeneric.Tests | | Error | 5 | Unable to find nupkg for System.Collections.Concurrent. | System.Collections.NonGeneric.Tests | | Error | 6 | Unable to find nupkg for System.Diagnostics.Tracing. | System.Collections.NonGeneric.Tests | | Error | 7 | Unable to find nupkg for System.IO.FileSystem. | System.Collections.NonGeneric.Tests | | Error | 8 | Unable to find nupkg for System.IO.FileSystem.Primitives. | System.Collections.NonGeneric.Tests | | Error | 9 | Unable to find nupkg for System.Linq. | System.Collections.NonGeneric.Tests | | Error | 10 | Unable to find nupkg for System.Reflection.Extensions. | System.Collections.NonGeneric.Tests | | Error | 11 | Unable to find nupkg for System.Runtime.Handles. | System.Collections.NonGeneric.Tests | | Error | 12 | Unable to find nupkg for System.Runtime.InteropServices. | System.Collections.NonGeneric.Tests | | Error | 13 | Unable to find nupkg for System.Text.Encoding.Extensions. | System.Collections.NonGeneric.Tests | | Error | 14 | Unable to find nupkg for System.Text.RegularExpressions. | System.Collections.NonGeneric.Tests | | Error | 15 | Unable to find nupkg for System.Xml.ReaderWriter. | System.Collections.NonGeneric.Tests | | Error | 16 | Unable to find nupkg for System.Xml.XDocument. | System.Collections.NonGeneric.Tests | 440 area-System.Text Initial commit of System.Text.Encoding.CodePages 442 area-System.Text Pull System.Text.Encoding.CodePages from v1.0 to master 445 area-System.Text Stop compiling Interop.cs into Unix build of code pages The Interop.cs file was being compiled into the unix build of System.Text.Encoding.CodePages. This commit just renames the file to be Windows-specific and includes it only in the Windows builds. It also does some minor cleanup of a couple of files. 447 area-Meta Update README.md to include System.IO.MemoryMappedFiles 449 area-Infrastructure Corefx build fails when Xamarin PCL profiles are installed 1. Install Xamarin tools 2. Build corefx EXPECTED: Build succeeds ACTUAL: Build fails with lots of type resolution errors The ResolveNuGetPackages target is failing to resolve assets out of xunit.core.netcore: ``` C:\git\corefx\src\packages\Microsoft.DotNet.BuildTools.1.0.21-prerelease\lib\packageresolve.targets(25,5): warning : Unable to resolve the assets of xunit.core.netcore: Couldn't find a matching group ``` The Xamarin tools add XML files to the SupportedFrameworks folders of some PCL profiles. This results in the configured frameworks for the project to be: ``` .NETFramework,Version=v4.5;.NETCore,Version=v4.5;WindowsPhoneApp,Version=v8.1;WindowsPhone,Version=v8.0;MonoAndroid,Version=v1.0;MonoTouch,Version=v1.0 ``` From the package, the following frameworks are supported: ``` .NETFramework,Version=v4.5;ASP.NetCore,Version=v5.0;.NETCore,Version=v4.5;WindowsPhoneApp,Version=v8.1;WindowsPhone,Version=v8.0 ``` So since the package doesn't support MonoTouch and MonoAndroid, the build system isn't resolving any of the assemblies inside it. 453 area-System.Xml Make fields readonly in System.Xml 454 area-System.Xml Using IsNullOrEmpty in checks Using IsNullOrEmpty in checks instead of direct checks on null or empty 455 area-Meta Consider merging labels `up-for-grabs` and `accepting prs` labels As discussed [here](https://github.com/dahlbyk/up-for-grabs.net/pull/146) it seems that these labels are semantically so close that it may make sense for us to merge them. - `up-for-grabs`. This is meant for tagging issues to indicate that we'd like the help of the community. - `accepting pr`. We use this label as part of the [API review process](https://github.com/dotnet/corefx/wiki/API-Review-Process). It's tagging proposals for API additions to indicate that we're happy to review & accept pull requests for those. It seems they are conceptually both doing the same thing: they are indicating to our community that we're happy if they jump in and help. Thoughts? 459 area-System.Linq Re-enable PLINQ tests The PLINQ tests are currently disabled. This PR turns them back on, alleviating the deadlocks that had caused them to be disabled initially. I also addressed a few other things, including some waits that weren't working correctly and some tests that should be OuterLoop due to their demands on the thread pool. On my machine, the tests as they are after the PR do take ~15 seconds to run as part of the build. I think that's completely reasonable for a set of tests for such a library, but if we're uncomfortable with that we can aim to bring it down further separately. For now, I just want to make sure there's at least some coverage in builds and CI. 461 area-System.Collections Add a static Shuffle method to Array and an instance method to List Consider adding a static Shuffle method to Array and an instance method to List ## Motivation Shuffling data is often required in various applications. .NET does not contain a method to shuffle an array which results in many people implementing it themselves. Implementing a Shuffle method is not really hard but it does require like 10 lines of code and people often wonder where to put these so they just prefer quick and dirty solutions like abusing sort methods with random sort key or even worse random comparison. These methods are far from ideal both because they depend on implementation details of the sorting methods and because they are less effective than the ideal O(N) implementation. ## Proposed API Methods on Array: ``` C# public static void Shuffle(T[] array) public static void Shuffle(T[] array, int index, int length) public static void Shuffle(T[] array, Random rng) public static void Shuffle(T[] array, int index, int length, Random rng) ``` Methods on List ``` C# public void Shuffle() public void Shuffle(Random rng) ``` ## Details The Shuffle method on Array has an overload that can be used to shuffle just part of the array. This is in line with existing Array methods like Sort and Copy and can be useful when implementing Shuffle on List where only the elements of the List will be shuffled without shuffling the full capacity of the list. I also suggest adding an instance Shuffle method on List since this is the most common usage of such a method. 464 area-System.ComponentModel Fix inverted documentation between two methods I just noticed this documentation mistake, the two method descriptions were swapped. 465 area-System.Linq Re-enable PLINQ tests Re-issuing #459 against v1.0 branch. Already reviewed there, so will merge once successfully passes CI. 466 area-Meta Update README.md for System.ComponentModel 467 area-System.Runtime Add a Clamp method to System.Math I know the `System.Math` library hasn't been pushed to the .Net Core library but since discussing api additions could take some time I thought it was a good idea to propose it now. # Rationale and Usage `Math.Clamp` is an easy to implement method and makes it easier for developers. Currently developers would have to implement their own methods in an extension class or use `Math.Max` and `Math.Min` together. The proposed API will shorten the amount of code needed and increase readability. Two examples on how we currently can clamp in .NET: ``` C# int result = Math.Max(Math.Min(value, 100), 0); ``` ``` C# public static class Ext { // Source: http://stackoverflow.com/a/2683487/1455541 public static T Clamp(T val, T min, T max) where T : IComparable { if (val.CompareTo(min) < 0) return min; else if(val.CompareTo(max) > 0) return max; else return val; } } // Somewhere in the codebase int result = Ext.Clamp(value, 0, 100); ``` The proposed API would turn that into: ``` C# int result = Math.Clamp(value, 0, 100); ``` # Proposed API 1 - Specialized ``` C# public static class Math { public static Byte Clamp(Byte value, Byte min, Byte max); public static Decimal Clamp(Decimal value, Decimal min, Decimal max); public static Double Clamp(Double value, Double min, Double max); public static Int16 Clamp(Int16 value, Int16 min, Int16 max); public static Int32 Clamp(Int32 value, Int32 min, Int32 max); public static Int64 Clamp(Int64 value, Int64 min, Int64 max); public static SByte Clamp(SByte value, SByte min, SByte max); public static Single Clamp(Single value, Single min, Single max); public static UInt16 Clamp(UInt16 value, UInt16 min, UInt16 max); public static UInt32 Clamp(UInt32 value, UInt32 min, UInt32 max); public static UInt64 Clamp(UInt64 value, UInt64 min, UInt64 max); } ``` This proposed version mimics the way current `System.Math` methods do it. # Proposed API 2 - IComparable Because this is a new API addition it can also be possible to have smaller implementation using generics that should cover all the cases. ``` C# public static class Math { public static T Clamp(T value, T min, T max) where T : System.IComparable; } ``` # Open Questions - If the api addition is going to be implemented should it be implemented as Proposed API 1 or 2? 468 area-System.Text Spelling fixes for System.Text.RegularExpressions Just fixing a handful of spelling errors in comments. 470 area-Meta Do not use optional arguments on public APIs As part of the [discussion](https://github.com/dotnet/apireviews/tree/master/2015-01-14-misc#110-add-async-documentelement-loading-for-xlinq) around adding async document/element loading for XLINQ (#110) it was mentioned using optional arguments (commonly called optional parameters). **PLEASE, DON'T USE IT ON PUBLIC APIS!!!** Before there were optional arguments the problem of a large number of possible parameters for an API already existed. And since we are talking about XML, look at how `XmlReader` and `XmlWritter` solved it: - [public static XmlReader Create(TextReader input, XmlReaderSettings settings)](http://referencesource.microsoft.com/#System.Xml/System/Xml/Core/XmlReader.cs,6375d7745e42dee9) - [public static XmlWriter Create(XmlWriter output, XmlWriterSettings settings)](http://referencesource.microsoft.com/System.Xml/a.html#c6b2420e91f21e33) Taking advantage of object initialization, calling it looks like optional arguments: ``` XmlReader.Create(reader, new XmlReaderSettings{ IgnoreWhitespace = true }); ``` (I'm going to propose more type inference for cases like this: [Do not require type specification for constructors when the type is known #35](https://github.com/dotnet/roslyn/issues/35)) It's extensible. Another setting can always be added with proper defaults and without breaking existing code. And those settings also become something that can be passed around. Imagine that the original API used optional arguments and there was only one `Save` method. How would you keep that and now add an `CancellationToken` parameter? 482 area-System.Xml Simple cleanup in XDocument classes 484 area-System.Xml "Explicitly marked XDocument private methods with ""private"" keyword" The `private` keyword was used for fields within the XDocument classes but not the method definitions. Updated for consistency. 489 area-System.Numerics Add primitive structs Rectangle and Point System.Drawing, System.Windows, and XNA all implement nearly-identical structures to represent points and rectangles. Given the verbosity of a correct implementation, many library developers re-use an existing implementation, and tie their public APIs to (non-core-compatible) dependencies. Offering primitives like Point, Rectangle (as well as PointF, RectangleF, and Color) would make it much easier for libraries to be compatible with each other. Or, is there a de-facto standard nuget package for these structs that we can promote? 490 area-System.Runtime Allow the Math methods to return float values I find myself casting returned results from Math methods into floats constantly, even if only floats are supplied as parameters for that method. Many developers don't need the accuracy of a double and floats works fine, many other numeric structs store their values are floats too (such as a Vector2). While explicit casting doesn't hurt anyone there are other classes I have come by that return a float if the supplied parameter(s) is float only. This wouldn't really be considered a new API addition, all it would do is add overloaded methods that accepts floats as parameters and returns a float (could call the Double version of the Method underneath to avoid code duplication and cast back). If any of the parameters become a double then the double version of the Method would be called and a double value would return. I would like feedback on what other people think before implementing it. 492 area-System.Collections statistic distribution generator I am thinking to add some random generators that obeys specific statistic distribution functions (like normal, binomial, geometric and so on) and also makes the random generator usable as IEnumerable to be easier to use in line or sequence expressions. There is a great paper that covers the ability to obtain sampling functions by composing other sampling functions that is quite nice to express via IEnumerable and IEnumerator. Suck functions can be very useful to obtain numbers that follow a specific law, for example to be used in generating correct weights to initialise untrained neural networks or similar scenarios. 494 area-System.Runtime Consider exposing extension methods for System.Array .Net 2 convenience methods With .net 2.0, `System.Array` was given a set of convenient LINQ like methods like those present on `List`. Unlike `List`, they were added as static methods. While, LINQ does offer some of these methods these variants do offer some differences/benefits as they are A: Optimized for arrays. B: Evaluated immediately. C: Always produce an array. These methods are:: ``` csharp AsReadOnly BinarySearch ConvertAll Exists Find FindAll FindIndex FindLast FindLastIndex ForEach IndexOf LastIndexOf Sort TrueForAll ``` Some of these offer no real benefit over LINQ, but others can be very useful especially in array heavy APIs like Reflection. Primarily `ConvertAll` and `FindAll`. These methods would also make it easier to work with arrays. There would be no clash with existing LINQ methods as they have different names. 496 area-System.Xml Reformat Xml code Guidelines were updated few times since the last time it was done. 497 area-Meta Remove non-cancellable overloads from async APIs I'd like to start discussion on removing the convenience overloads that call a cancellable API with `CancellationToken.None`: `Task DoAsync() { return DoAsync(CancellationToken.None); }` I think these overloads bloat the API and if you've got a large number of parameters, can be error-prone to implement. I also believe they encourage poor design: consider that a great majority of async code either already does or should support cancellation. The use of these APIs should be relatively rare in good code. 498 area-System.Collections HashSet constructor performance tuning for ISet "As it is currently, `HashSet` has a constructor that takes `IEnumerable`. `var set1 = new HashSet{2,3,4};` `var set2 = new HashSet(set1); // binds against ctor..(IEnumerable)` This works nicely, but the fact that we are ""copying/cloning"" from `ISet` is not being taken advantange of. If we have `HashSet` with ~10000 elements, **there is no efficient way to create a copy of it**. Instead, current implementation will call internally `AddIfNotPresent` for each element in given `IEnumerable`, eg, do an expensive element lookup for each item that is being added (to check if it doesn't exist in our current `HashSet`). This does not have to be the case. `ISet` already promises us the uniqueness. Let me know if this is something of interest, I would like to work on that. " 503 area-Meta Make use of Code Contracts where ever applicable. I wonder the reason why while the code contracts are part of BCL and it is a research project of Microsoft, only some of the API makes use of it. This this is effectively preventing making use of code contracts for the end consumers as well. I would appreciate at least if you can make use of Code Contracts on the public surface API. 507 area-System.Collections Specification tests for collection interfaces In the last [API review](https://github.com/dotnet/apireviews/tree/master/2015-01-14-misc#follow-ups-for-api-review-board), @KrzysztofCwalina mentioned that it would be nice to have a set of tests that can serve as specifications for the expected behavior of the built-in collection interfaces. These tests could be used by people creating types which implement these interfaces to ensure that their implementations match the specification. I love this idea, and would love to help create the suite. Is this something that the community can engage on? 516 area-Meta Replace hand-rolled tests for exceptions with xunit's Assert.Throws There is a lot of hand-rolled exception checking in various test code. This should be refactored to use xunit's Assert.Throws method as it improves readability and can reduce possible bugs in test code. For an example please see PR https://github.com/dotnet/corefx/pull/512 519 area-System.Runtime Allow BitConverter.ToString to format without dashes "I think BitConverter is not even in CoreFx yet, but for when it is, please allow to convert to an hex string without dashes. This is to avoid this pattern: ``` BitConverter.ToString(data).Replace(""-"", string.Empty); ``` This representation of bytes is common enough that .NET should have a simple method to do this without extra code or an extra memory allocation. " 522 area-System.Threading Thread apartment states in .NET Core Hi all! Just wondering, will you support thread apartment state in .NET Core ? Currently there are no support. Maybe in future? Thanks. 525 area-Meta Consider creating a gitter channel for corefx repo "I was checking out the Aurelia docs (new JS framework), and they casually mention ""If you have questions, we hope that you will join us on our [gitter channel](https://gitter.im/Aurelia/Discuss)."" Gitter is apparently a free group chat for open source projects hosted on github. Here's the best ""About"" page I could find: http://blog.gitter.im/about/. It looks like they have really good github integration (an activity feed that shows PRs; issues; commits; etc). I'd be really interested in trying out a gitter channel for the corefx repo. " 526 area-System.Collections Dictionary class should allow to use a custom load factor Hello guys, I had seen the `Dictionary` class does not allow the use of a custom `load factor`, this may allow many optimizations, as the case we want to create a dictionary of known initial size and it will be completely filled, but we don't want that resizing operations occurs, for example it would be possible setting the `load factor` to `1.0F`. The class constructors below may be added: ``` csharp public Dictionary(int capacity, float loadFactor); public Dictionary(int capacity, float loadFactor, IEqualityComparer comparer) ``` Simirlarly to `Java`'s `HashMap` class constructor: ``` java public HashMap(int initialCapacity, float loadFactor); ``` When the `load factor` is not known, it should take a default value, for example `0.75F`. Thanks a lot :smile: and excuse me for any inconvenience. 529 area-Infrastructure Clean target removes dependencies "The ""Clean"" target of the build script intentionally removes the ""src\packages"" directory, causing the next build to have to restore them again. This is unnecessary, and atypical compared to most build scripts. ""Clean"" typically removes the outputs of the build process, not the inputs. If there is a need for an ability to wipe out the packages directory, I propose putting this functionality under a separate target, ""CleanDependencies"". " 537 area-System.Globalization Refactor globalization extensions with PAL Simple refactoring to introduce a PAL layer into System.Globalization.Extensions. Cleaned up a few things along the way as well, e.g. factoring out some duplicated code into helpers, making a few fields readonly, etc. 538 area-System.Runtime New API proposal: null replacement with Option, Some and None "The .NET framework and C# have implemented a lot of ideas of functional programming. LINQ, lambdas and immutability are only a few examples. In my eyes this makes the real difference between Java and .NET Solving the [null reference problem](http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare) would be the next great addition to the .NET framework borrowed from functional languages. So in this issue I want to propose the addition of a [Option type](http://en.wikipedia.org/wiki/Option_type) to the .NET Framework. # Background Taken from https://github.com/louthy/language-ext#null-reference-problem: > null must be the biggest mistake in the whole of computer language history. I realise the original designers of C# had to make pragmatic decisions, it's a shame this one slipped through though. So, what to do about the 'null problem'? > > null is often used to indicate 'no value'. i.e. the method called can't produce a value of the type it said it was going to produce, and therefore it gives you 'no value'. The thing is that when 'no value' is passed to the consuming code, it gets assigned to a variable of type T, the same type that the function said it was going to return, except this variable now has a timebomb in it. You must continually check if the value is null, if it's passed around it must be checked too. > > Option works in a very similar way to Nullable except it works with all types rather than just value types. It's a struct and therefore can't be null. An instance can be created by either calling Some(value), which represents a positive 'I have a value' response; Or None, which is the equivalent of returning null. > > So why is it any better than returning T and using null? It seems we can have a non-value response again right? Yes, that's true, however you're forced to acknowledge that fact, and write code to handle both possible outcomes because you can't get to the underlying value without acknowledging the possibility of the two states that the value could be in. This bulletproofs your code. You're also explicitly telling any other programmers that: ""This method might not return a value, make sure you deal with that"". This explicit declaration is very powerful. # Rationale and Usage So basically the `Option` type enforces developers to check if a return value has really a value. An example function returning an option would be something like this: ``` public Option GetCacheValue(string key) { if(cache.Keys.Contains(key)) { return cache[key]; //return new Some(cache[key]) //return new Option(cache[key]); } else { return Option.None; //return new Option(); } } ``` The consumer code would then look like this: ``` string value = GetCacheValue(key).Match( some: v => v, none: () => ""this is the default value"" ) ``` # Proposed API ``` public struct Option { // For creating a None public Option(); public Option(T value) public static readonly Option None = new Option(); public bool IsNone {get; }; public bool IsSome { get; } public static implicit operator Option(T value) public static implicit operator Option(OptionNone none) public TReturn Match(Func some, Func none); public static Option Some(T value); } ``` # Details - This was mainly inspired by https://github.com/louthy/language-ext/blob/master/LanguageExt.Core/Option.cs of @louthy # Open questions - [ ] Should there be a public `Value` property as an alternative to Match? - [ ] In which assembly would this be correct? Namespace? - [ ] Should Option be a struct (security, since it cannot be null) or an interface (usability, since `Option` is then also an `Option`)? Thoughts? " 539 area-Meta Improve flow of readme Changed the ordering of content and a few pieces of wording. Primary goal was to pop contributing opportunities as early as possible in the readme, since the components list is getting quite long. We may want to do something different with the list as it grows even longer, but we'll not worry about that now. 540 area-System.Xml Remove double type checks in System.Xml Replace code like this: ``` if (obj is Foo) ((Foo)obj).DoSomething(); ``` with ``` Foo foo = obj as Foo; if (foo != null) foo.DoSomething(); ``` Most of these changes were verified in #491 and #482, but these PRs were reverted due reasons non-related with these changes. 549 area-System.IO DriveInfo tests fail if current user uses SUBST On my machine I'm using SUBST to map folders as drives. This causes the the drive info tests to fail: ``` System.IO.FileSystem.DriveInfoTests.Get_VolumeLabel.Test01 [FAIL] System.IO.IOException : The directory is not a subdirectory of the root directory. Stack Trace: p:\oss\corefx\src\System.IO.FileSystem.DriveInfo\src\System\IO\DriveInfo.Windows.cs(235,0): at System.IO.DriveInfo.set_VolumeLabel(String value) p:\oss\corefx\src\System.IO.FileSystem.DriveInfo\tests\GetVolumeLabelTests.cs(74,0): at System.IO.FileSystem.DriveInfoTests.Get_VolumeLabel.Test01() System.IO.FileSystem.DriveInfoTests.Set_VolumeLabel.Test01 [FAIL] System.IO.IOException : The directory is not a subdirectory of the root directory. Stack Trace: p:\oss\corefx\src\System.IO.FileSystem.DriveInfo\src\System\IO\DriveInfo.Windows.cs(235,0): at System.IO.DriveInfo.set_VolumeLabel(String value) p:\oss\corefx\src\System.IO.FileSystem.DriveInfo\tests\SetVolumeLabelTests.cs(75,0): at System.IO.FileSystem.DriveInfoTests.Set_VolumeLabel.Test01() ``` I would expect the drive info tests to be resilient to this. 572 area-System.Console Initial commit of System.Console tests 574 area-System.Collections Add PriorityQueue to Collections "See [**LATEST Proposal**](https://github.com/dotnet/corefxlab/blob/master/docs/specs/priority-queue.md) in corefxlab repo. # Second Proposal options Proposal from https://github.com/dotnet/corefx/issues/574#issuecomment-307971397 ### Assumptions Elements in priority queue are unique. If they are not, we would have to introduce 'handles' of items to enable their update/remove. Or the update/remove semantics would have to apply to first/all, which is weird. Modeled after [`Queue`](https://apisof.net/catalog/System.Collections.Generic.Queue%3CT%3E) ([MSDN link](https://msdn.microsoft.com/en-us/library/7977ey2c(v=vs.110).aspx)) ### API ```c# public class PriorityQueue : IEnumerable, IEnumerable<(TElement element, TPriority priority)>, IReadOnlyCollection<(TElement element, TPriority priority)> // ICollection not included on purpose { public PriorityQueue(); public PriorityQueue(IComparer comparer); public IComparer Comparer { get; } public int Count { get; } public bool IsEmpty { get; } public bool Contains(TElement element); // Peek & Dequeue public (TElement element, TPriority priority) Peek(); // Throws if empty public (TElement element, TPriority priority) Dequeue(); // Throws if empty public bool TryPeek(out TElement element, out TPriority priority); // Returns false if empty public bool TryDequeue(out TElement element, out TPriority priority); // Returns false if empty // Enqueue & Update public void Enqueue(TElement element, TPriority priority); // Throws if it is duplicate public void Update(TElement element, TPriority priority); // Throws if element does not exist public void EnqueueOrUpdate(TElement element, TPriority priority); public bool TryEnqueue(TElement element, TPriority priority); // Returns false if it is duplicate (does NOT update it) public bool TryUpdate(TElement element, TPriority priority); // Returns false if element does not exist (does NOT add it) public void Remove(TElement element); // Throws if element does not exist public bool TryRemove(TElement element); // Returns false if element does not exist public void Clear(); public IEnumerator<(TElement element, TPriority priority)> GetEnumerator(); IEnumerator IEnumerable.GetEnumerator(); // // Selector part // public PriorityQueue(Func prioritySelector); public PriorityQueue(Func prioritySelector, IComparer comparer); public Func PrioritySelector { get; } public void Enqueue(TElement element); public void Update(TElement element); } ```` **Open questions:** 1. Class name `PriorityQueue` vs. `Heap` 2. Introduce `IHeap` and constructor overload? (Should we wait for later?) 3. Introduce `IPriorityQueue`? (Should we wait for later - `IDictionary` example) 4. Use selector (of priority stored inside the value) or not (5 APIs difference) 5. Use tuples `(TElement element, TPriority priority)` vs. `KeyValuePair` * Should `Peek` and `Dequeue` rather have `out` argument instead of tuple? 6. Is `Peek` and `Dequeue` throwing useful at all? ----------------------------------------------------------------------------------------------- # Original Proposal Issue https://github.com/dotnet/corefx/issues/163 requested the addition of a priority queue to the core .NET collection data structures. This post, while a duplicate, is intended to act the formal submission to the corefx API Review Process. The issue contents are the _speclet_ for a new System.Collections.Generic.PriorityQueue type. I will be contributing the PR, if approved. ## Rationale and Usage The .NET Base Class Libraries (BCL) currently lacks support for ordered producer-consumer collections. A common requirement of many software applications is the ability generate a list of items over time and process them in an order different from the order they were received in. There are three generic data structures within the System.Collections hierarchy of namespaces that supported a sorted collection of items; System.Collections.Generic.SortedList, System.Collections.Generic.SortedSet, and System.Collections.Generic.SortedDictionary. Of these, SortedSet and SortedDictionary are not appropriate for producer-consumer patterns that generate duplicate values. The complexity of SortedList is Θ(n) worst case for both Add and Remove. A much more memory and time efficient data structure for ordered collections with producer-consumer usage patterns is a priority queue. Other than when capacity resizing is necessary, worse case insertion (enqueue) and remove top (dequeue) performance is Θ(log n) - far better than the existing options that exist in the BCL. Priority queues have a wide degree of applicability across different classes of applications. The Wikipedia page on Priority Queues offers a list of many different well understand use cases. While highly specialized implementations may still require custom priority queue implementations, a standard implementation would cover a broad range of usage scenarios. Priority queues are particularly useful in scheduling the output of multiple producers, which is an important pattern in highly parallelized software. It's worth noting that both the C++ standard library and Java offer priority queue functionality as part of their basic APIs. ## Proposed API ``` C# namespace System.Collections.Generic { /// /// Represents a collection of objects that are removed in a sorted order. /// /// Specifies the type of elements in the queue. [DebuggerDisplay(""Count = {count}"")] [DebuggerTypeProxy(typeof(System_PriorityQueueDebugView<>))] public class PriorityQueue : IEnumerable, ICollection, IEnumerable, IReadOnlyCollection { /// /// Initializes a new instance of the class /// that uses a default comparer. /// public PriorityQueue(); /// /// Initializes a new instance of the class /// that has the specified initial capacity. /// /// The initial number of elements that the can contain. /// is less than zero. public PriorityQueue(int capacity); /// /// Initializes a new instance of the class /// that uses a specified comparer. /// /// The to use when comparing elements. /// is null. public PriorityQueue(IComparer comparer); /// /// Initializes a new instance of the class /// that contains elements copied from the specified collection and uses a default comparer. /// /// The collection whose elements are copied to the new . /// is null. public PriorityQueue(IEnumerable collection); /// /// Initializes a new instance of the class /// that contains elements copied from the specified collection and uses a specified comparer. /// /// The collection whose elements are copied to the new . /// The to use when comparing elements. /// /// is null. -or- /// is null. /// public PriorityQueue(IEnumerable collection, IComparer comparer); /// /// Initializes a new instance of the class that is empty, /// has the specified initial capacity, and uses a specified comparer. /// /// The initial number of elements that the can contain. /// The to use when comparing elements. /// is less than zero. /// is null. public PriorityQueue(int capacity, IComparer comparer); /// /// Gets the for the . /// /// /// The that is used when /// comparing elements in the . /// public IComparer Comparer { get; } /// /// Gets the number of elements contained in the . /// /// The number of elements contained in the . public int Count { get; } /// /// Adds an object to the into the by its priority. /// /// /// The object to add to the . /// The value can be null for reference types. /// public void Enqueue(T item); /// /// Removes and returns the object with the lowest priority in the . /// /// The object with the lowest priority that is removed from the . /// The is empty. public T Dequeue(); /// /// Returns the object with the lowest priority in the . /// /// The is empty. public T Peek(); /// /// Removes all elements from the . /// public void Clear(); /// /// Determines whether an element is in the . /// /// /// The object to add to the end of the . /// The value can be null for reference types. /// /// /// true if item is found in the ; otherwise, false. /// public bool Contains(T item); /// /// Copies the elements of the to an , /// starting at a particular index. /// /// /// The one-dimensional Array that is the /// destination of the elements copied from the . /// The Array must have zero-based indexing. /// /// The zero-based index in at which copying begins. /// is null. /// /// is less than zero. -or- /// is equal to or greater than the length of the /// /// /// The number of elements in the source is /// greater than the available space from to the end of the destination /// . /// public void CopyTo(T[] array, int arrayIndex); /// /// Copies the elements of the to an /// , starting at a particular index. /// /// /// The one-dimensional Array that is the /// destination of the elements copied from the . /// The Array must have zero-based indexing. /// /// The zero-based index in at which copying begins. /// is null. /// is less than zero. /// /// is multidimensional. -or- /// does not have zero-based indexing. -or- /// is equal to or greater than the length of the -or- /// The number of elements in the source is /// greater than the available space from to the end of the destination /// . -or- /// The type of the source cannot be cast automatically /// to the type of the destination . /// void ICollection.CopyTo(Array array, int index); /// /// Copies the elements stored in the to a new array. /// /// /// A new array containing a snapshot of elements copied from the . /// public T[] ToArray(); /// /// Returns an enumerator that iterates through the /// /// An enumerator for the contents of the . public Enumerator GetEnumerator(); /// /// Returns an enumerator that iterates through the /// /// An enumerator for the contents of the . IEnumerator IEnumerable.GetEnumerator(); /// /// Returns an enumerator that iterates through the . /// /// An that can be used to iterate through the collection. IEnumerator IEnumerable.GetEnumerator(); /// /// Sets the capacity to the actual number of elements in the , /// if that number is less than than a threshold value. /// public void TrimExcess(); /// /// Gets a value that indicates whether access to the is /// synchronized with the SyncRoot. /// /// true if access to the is synchronized /// with the SyncRoot; otherwise, false. For , this property always /// returns false. bool ICollection.IsSynchronized { get; } /// /// Gets an object that can be used to synchronize access to the /// . /// /// /// An object that can be used to synchronize access to the /// . /// object ICollection.SyncRoot { get; } public struct Enumerator : IEnumerator { public T Current { get; } object IEnumerator.Current { get; } public bool MoveNext(); public void Reset(); public void Dispose(); } } } ``` ## Details - Implementation data structure will be a binary heap. Items with a greater comparison value will be returned first. (descending order) - Time complexities: | Operation | Complexity | Notes | | --- | --- | --- | | Construct | Θ(1) | | | Construct Using IEnumerable | Θ(n) | | | Enqueue | Θ(log n) | | | Dequeue | Θ(log n) | | | Peek | Θ(1) | | | Count | Θ(1) | | | Clear | Θ(N) | | | Contains | Θ(N) | | | CopyTo | Θ(N) | Uses Array.Copy, actual complexity may be lower | | ToArray | Θ(N) | Uses Array.Copy, actual complexity may be lower | | GetEnumerator | Θ(1) | | | Enumerator.MoveNext | Θ(1) | | - Additional constructor overloads that take the System.Comparison delegate were intentionally omitted in favor of a simplified API surface area. Callers can use Comparer.Create to convert a function or Lambda expression to an IComparer interface if necessary. This does require the caller to incur a one-time heap allocation. - Although System.Collections.Generic is not yet part of corefx, I propose that this class be added to corefxlab in the meantime. It can be moved to the primary corefx repository once System.Collections.Generic are added and there is consensus that its status should be elevated from experimental to an official API. - An IsEmpty property was not included, since there is no additional performance penalty calling Count. The majority of the collection data structures do not include IsEmpty. - The IsSynchronized and SyncRoot properties of ICollection were implemented explicitly as they are effectively obsolete. This also follows the pattern used for the other System.Collection.Generic data structures. - Dequeue and Peek throw an InvalidOperationException when the queue is empty to match the established behavior of System.Collections.Queue. - IProducerConsumerCollection was not implemented as its documentation states that it is only intended for thread-safe collections. ## Open Questions - Is avoiding an additional heap allocation during calls to GetEnumerator when using foreach a strong enough rationale for including the nested public enumerator structure? - Should CopyTo, ToArray, and GetEnumerator return results in prioritized (sorted) order, or the internal order used by the data structure? My assumption is that the internal order should be returned, as it doesn't incur any additional performance penalties. However, this is a potential usability issue if a developer thinks of the class as a ""sorted queue"" rather a priority queue. - Does adding a type named PriorityQueue to System.Collections.Generic cause a potentially breaking change? The namespace is heavily used, and could cause a source compatibility problem for projects that include their own priority queue type. - Should items be dequeued in ascending or descending order, based on the output of IComparer? (my assumption is ascending order, to match the normal sorting convention of IComparer). - Should the collection be 'stable'? In other words, should two items with equal IComparison results be dequeued in the exact same order they are enqueued in? (my assumption is this isn't needed) ## Updates - Fixed complexity of 'Construct Using IEnumerable' to Θ(n). Thanks @svick. - Added another option question regarding whether the priority queue should be ordered in ascending or descending order compared to the IComparer. - Removed NotSupportedException from explicit SyncRoot property to match behavior of other System.Collection.Generic types instead of using the newer pattern. - Made the public GetEnumerator method return a nested Enumerator struct instead of IEnumerable, similar to the existing System.Collections.Generic types. This is an optimization to avoid a heap (GC) allocation when using a foreach loop. - Removed ComVisible attribute. - Changed complexity of Clear to Θ(n). Thanks @mbeidler. " 579 area-System.Globalization System.Globalization.Extensions.Tests fail on Windows 7 "On Win7 (7601) the globalization extensions have several failures: System.Globalization.Extensions.Tests.IdnaConformanceTests.TestAsciiPositive [FAIL] System.ArgumentException : Decoded string is not a valid IDN name. Parameter name: Unicode System.Globalization.Extensions.Tests.IdnaConformanceTests.TestUnicodePositive [FAIL] Expected: <+á.+ɦê>. Actual:. Error on line number 61 System.Globalization.Extensions.Tests.IdnaConformanceTests.TestAsciiNegative [FAIL] Assert.Throws() Failure Expected: typeof(System.ArgumentException) Actual: (No exception was thrown) System.Globalization.Extensions.Tests.NormalizationAll.NormalizeTest [FAIL] Assert.False() Failure (many lines like ""'\x0068' is not matched with the normalized form '\x2095 with Géò normalization"") " 581 area-System.Xml Improve performance and allocations of xpath string functions "Improve Translate perf by ~20% and allocations by around 50% (dependant on string) Improve Normalize perf by ~30% and allocations by around 55% (dependant on string) General: We are always iterating through every char so we can get them all in one go and use that as our temporary buffer instead of the StringBuilder. Because these functions can remove chars there is a read point in the buffer and an update point. The update point is always the same place or less than the read point. Normalize: No need for XmlConvertEx.TrimString as we can start ""firstSpace"" as false to trim the start and all trailing whitespace at the end is combined into one space. We can just check that at the end which avoids a trim string creation. " 584 area-System.Linq ParallelEnumerable.Range(...).AsOrdered().Concat(...) ...out of order I'm getting a pretty distressing error: ``` int leftSize = 4; int rightSize = 4; int[] leftData = new int[leftSize]; for (int i = 0; i < leftSize; i++) leftData[i] = i; int[] rightData = new int[rightSize]; for (int i = 0; i < rightSize; i++) rightData[i] = i; ParallelQuery query = ParallelEnumerable.Range(0, leftSize).AsOrdered().Concat(ParallelEnumerable.Range(0, rightSize)); Assert.True(leftData.Concat(rightData).SequenceEqual(query)); ``` ...fails. The expected sequence is `[0, 1, 2, 3, 0, 1, 2, 3]`, and the actual result of this `Concat(...)` is `[0, 0, 1, 1, 2, 2, 3, 3]`. Adding `AsOrdered()` to the inner range has no effect. There is no history of changes for `Concat(...)`, although the actual problem may be elsewhere. This issue is **NOT** present in the current .NET distribution (ie, creating a brand-new C# project won't reproduce the error, this is something happening in corefx, which is a really good thing)/ Generating the parallel sequence via `Enumerable.Range(...).AsParallel().AsOrdered()` _also_ fails, but seems to be putting the elements in non-deterministic order (the last run was `[0, 1, 1, 2, 2, 3, 0, 3]`). However, generating the sequence via `leftData.AsParallel().AsOrdered().Concat(leftData.AsParallel())` returns the expected sequence (both of them have to be generated via backing arrays, or you get the re-ordered or non-deterministic results). `ParallelEnumerable.Range(...).AsOrdered()` _does_ return an ordered sequence. The following test succeeds: ``` int count = 8; ParallelQuery elements = ParallelEnumerable.Range(0, count).AsOrdered().Select(x => (object)x); int counter = 0; Assert.All(elements.Cast(), x => Assert.Equal(counter++, x)); Assert.Equal(count, counter); ``` 585 area-System.Linq Fix PLINQ concat ordering bug For .NET Core, some (temporary) changes were made to PLINQ to remove some of its usage of generics in order to workaround some (temporary) constraints of the runtime. In doing so, this bug slipped in, with the wrong concat key being created for an element in the sequence, causing erroneous results. This PR fixes the bug and adds a test case that fails before and succeeds after the fix. (Fixes #584) 589 area-System.Linq Address remaining PLINQ test ActiveIssues A bunch of PLINQ tests are disabled as ActiveIssue. This commit fixes all of them (issues #176 and #240), except for #235, which is addressed by a separate PR (#585) for a product bug rather than test issues. I did not do a general pass through the PLINQ tests, even though they could use it. PlinqModesTests - There's an explicit comment at the top of the test that it's not meant for DOP==1 (it verifies that work happens in parallel), and then apparently we were surprised when it was failing on single core machines that result in DOP == 1 ;) UnionIntersectDistinctTests.RunOreredUnionTest1 - There's no reason this test should fail. If it does, there's a bug to be investigated. I'm assuming it was actually stalling because of another test running with it concurrently. WithCancellationTests - A bunch of these tests were scheduling tasks to asynchronously cancel/dispose a query, and the query was written to take a certain amount of time to hopefully allow for cancellation to take place. But it appears as part of the tests being ported, those timeouts were simply commented out, making it extremely likely the tests would fail. I've just rewritten them to do the cancellation as part of the query's execution rather than using a separate asynchronous task. I also deleted some duplicative tests. And I made some tests that still did explicit delays/timeouts to be OuterLoop. 594 area-System.Runtime Parsing string to enum with digit as first char takes a very long time "Note: This has to do with mscorlib. https://github.com/Microsoft/referencesource sent me here, but this doesn't look like the place where this is handled. Am I right? When parsing strings to enums with TryParse, in some cases this takes way longer than in all the other cases. After a bit of research I found that all the strings with a digit as first char are slower. This is due to the attempt to parse the string to the underlying type (https://github.com/Microsoft/referencesource/blob/master/mscorlib/system/enum.cs#L441) with a Convert.ChangeType, which can throw an exception, because further down the line it uses (for example) Int32.Parse instead of TryParse (https://github.com/Microsoft/referencesource/blob/master/mscorlib/system/convert.cs#L1121). The time it takes to create the Exception is significant. In my project I ended up writing a new Enum parser to circumvent this issue. ""Something"": 240.49 ns per parse ""1Somethin"": 37178.57 ns per parse ""123456789"": 485.63 ns per parse As users expect Convert.ToXxx() to throw an exception when it fails, changing this is a bad idea I think. Possible solutions I can think of: - Extend the Convert class with Try-methods that don't throw exceptions, and use those while trying to parse an enum (much work) - Instead of checking whether the first character is a digit (https://github.com/Microsoft/referencesource/blob/master/mscorlib/system/enum.cs#L434), change that line to a TryParse of the whole string (the easier way) " 598 area-Meta Improve flow of readme The readme file contains several pieces of information, of which the list of components is the most substantial. I pulled the engagement-oriented content above the component list, to make it easier to read. I also attempted to better clarify and position CoreFX within .NET Core. At the point that CoreCLR shows up, this will be more important. Supersedes PR #539, which had the wrong baseline and targeted the wrong branch. 617 area-System.Diagnostics Failure in System.Diagnostics.FileVersionInfoTest on Chinese Windows 8.1 "Four out of the six tests in System.Diagnostics.FileVersionInfoTest failed when `build` on Chinese Windows 8.1. They might also fail on other non-ENU OS'es. - FileVersionInfoTest.FileVersionInfo_Normal [FAIL] Error - Property 'Language' incorrect. Expected == ""English (United States)"", Actual == ""\u82f1\u8bed(\u7f8e\u56fd)"", Alternate == <null> - FileVersionInfoTest.FileVersionInfo_Chinese [FAIL] Error - Property 'Language' incorrect. Expected == ""Chinese (Simplified)"", Actual == ""\u4e2d\u6587(\u7b80\u4f53)"", Alternate == ""Chinese (Simplified, PRC)"" - FileVersionInfoTest.FileVersionInfo_DifferentFileVersionAndProductVersion [FAIL] Error - Property 'Language' incorrect. Expected == ""Process Default Language"", Actual == ""\u8fdb\u7a0b\u9ed8\u8ba4\u8bed\u8a00"", Alternate == <null> > - FileVersionInfoTest.FileVersionInfo_CustomManagedAssembly [FAIL] Error - Property 'Language' incorrect. Expected == ""Language Neutral"", Actual == ""\u8bed\u8a00\u4e2d\u6027"", Alternate == <null> " 621 area-Infrastructure Multiple nuget src\ dirs? Why are there multiple nuget src dirs.? There is a .nuget with a single file and a nuget with multiple files. Was this an oversight? Can I fix/combine them? 624 area-System.Diagnostics FileVersionInfo test: Get expected values using VerLanguageName call #617 It seems that VerLanguageName returns value in Windows locale no matter what thread locale is set, so must get expected values using that function. Fixes #617 Also, as really minor addition - change newline from \r\n to Environment.NewLine to match with same change in FileVersionInfo class (since it now uses AppendLine) 625 area-Meta HelloWorld WinRT Dependency? Hello, I'm trying to build the HelloWorld demo based off the Blog post instructions. It looks like the nuget.org packages aren't quite synced up, so that's a shame and I'm using 22416 instead. After I build and attempt to run the program I get an error stating that winrt-string is missing. Should this be working on Windows 7? -Thanks 626 area-System.Runtime Deprecate DateTime.Now (The original issue was at dotnet/coreclr#45), but I've moved it over here as the meat of the issue is about framework design. From @mj1856 I would like to propose that [`DateTime.Now`](https://github.com/dotnet/coreclr/blob/ef1e2ab328087c61a6878c1e84f4fc5d710aebce/src/mscorlib/src/System/DateTime.cs#L904-L920) be deprecated. This is most developer's first introduction to working with time, and is also a root cause of bugs related to daylight saving time and time zones. Developers should get in the habit of thinking globally, rather than locally - especially when designing for the web. Even on desktop and mobile applications that only run in a single time zone, this can create the kind of bugs that pull developers out of bed at 2AM on the morning of a daylight saving time transition. More supporting arguments: - [The Case Against DateTime.Now](http://www.codeofmatt.com/2013/04/25/the-case-against-datetime-now/) - Matt Johnson (me) - [What's Wrong with DateTime Anyway](http://blog.nodatime.org/2011/08/what-wrong-with-datetime-anyway.html) - Jon Skeet Given the widespread nature of this API, I suggest _not_ removing it from coreclr - but rather marking it with an `[Obsolete]` attribute. I can send a PR if the suggestion is approved. 636 area-System.Diagnostics Possible typo in FileVersionInfoTest "This is low priority, but logging here so it doesn't get lost. In [FileVersionInfoTest.cs line 239](https://github.com/dotnet/corefx/blob/0252409de5d77d4a104903a68d9ee34d54992e54/src/System.Diagnostics.FileVersionInfo/tests/FileVersionInfoTest.cs#L239): ``` s_fviSecondNativeLibrary.ProductName = ""Unkown_Product_Name""; ``` ""Unkown"" should be ""Unknown"", but correcting it fails the test. It should probably be first corrected wherever it originates from. Thanks. " 641 area-System.Xml Some System.Xml.Linq.XDocument tests are failing Some of the tests used to print error messages but did not fail. I added Asserts assuming that they fail and created this issue. I did not check if this is a test bug, product bug or not a bug at all (we need to check what happens on desktop and if this is something we should fix). 642 area-System.Xml Clean up System.Xml.Linq.XDocument tests "- [x] Remove spam messages (there are few left) - [x] Some of the tests might be failing but not throwing (look at the messages - https://github.com/dotnet/corefx/issues/641) - [x] Convert tests to use Xunit (right now they are only partially) - [x] Replace if (expr) throw new TestException, etc with proper use of Xunit asserts " 644 area-System.Reflection Proposal: Additional methods to aid in reflection of generic types "I don't know if this is the appropriate place to make such a suggestion. ## Problem I've always found it a bit annoying that the reflection APIs in .NET haven't been improved much since the advent of generics and when working with generic types or methods that considerable additional boilerplate needs to be written in order to correctly work with these types. ## Solution In the past I've implemented helper extension methods which significantly simplify reflection over generic types and methods. These extension methods largely work on top of the public surface of the `System.Type` class, although a few dealing with generic methods did resort to reflection in order to call internal methods to better detect and deal with ambiguities. I think that it would be very convenient to have these methods (or something like them) considered for addition into the core framework. I am willing and able to share my extension methods and could start a branch if there is traction to this idea. ### FindGenericInterfaces Identical to `Type.FindInterfaces` except that it will match on open generic interface types. ``` cs public delegate bool GenericTypeFilter(Type type, Type[] genericArguments, object filterCriteria); public static Type[] FindGenericInterfaces(this Type type, Type genericInterfaceType); public static Type[] FindGenericInterfaces(this Type type, Type genericInterfaceType, GenericTypeFilter filter, object filterCriteria); Type[] interfaces = typeof(int[]).FindGenericInterfaces(typeof(IEnumerable<>), (type, types, criteria) => types[0] == typeof(int), null); ``` ### ImplementsInterface Returns `true` if the type implements the specified interface. The interface can either be an open generic type or a closed constructed type. If the case of the former the overloads can be used to return the generic type arguments. For those overloads that accept a specific count of `out` parameters the generic type arguments must match the count of the `out Type` arguments. For the overload that accepts a `Type[]` the number of generic type arguments must match the length of the array. The overload that accepts an `out Type[]` will match any number of generic type arguments. _I also have `IsSubclassOf` and `Is` extension methods that function in almost the same manner except that the first matches if the type is a subclass of the specified type and the second matches if the type is a subclass of, implements or is of the specified type._ ``` cs public static bool ImplementsInterface(this Type type, Type interfaceType); public static bool ImplementsInterface(this Type type, Type interfaceType, out Type genericArgument); public static bool ImplementsInterface(this Type type, Type interfaceType, out Type genericArgument1, out Type genericArgument2); public static bool ImplementsInterface(this Type type, Type interfaceType, out Type genericArgument1, out Type genericArgument2, out Type genericArgument3); public static bool ImplementsInterface(this Type type, Type interfaceType, Type[] genericArguments); public static bool ImplementsInterface(this Type type, Type interfaceType, out Type[] genericArguments); Type elementType, keyType; if (typeof(int[]).ImplementsInterface(typeof(IEnumerable)) { ... } if (typeof(int[]).ImplementsInterface(typeof(IEnumerable<>), out elementType)) { ... } if (typeof(Dictionary).ImplementsInterface(typeof(IDictionary<>), out keyType, out elementType)) { ... } Type[] arguments = new Type[2]; if (typeof(Dictionary).ImplementsInterface(typeof(IDictionary<>), arguments)) { ... } arguments = null; if (typeof(Dictionary).ImplementsInterface(typeof(IDictionary<>), out arguments)) { ... } ``` ### GetGenericMethod _Separated into proposal [Add GetGenericMethod method to simplify finding generic methods via reflection](https://github.com/dotnet/corefx/issues/16567)._ Returns the generic method with the same number of specified generic arguments. The generic arguments are checked for compatibility with the generic type constraints. If a parameter type list is also specified then the parameter types must be acceptable for the given generic type arguments. _In order to properly handle matches and ambiguity resolution I did have to break out reflection in order to invoke the internal members `RuntimeType.FilterApplyMethodBase`, `DefaultBinder.FindMostDerivedNewSlotMeth` and `DefaultBinder.CompareMethodSigAndName`. A built-in implementation would not require to rely on any reflection._ ``` cs public static MethodInfo GetGenericMethod(this Type type, string name, Type[] genericArgs); public static MethodInfo GetGenericMethod(this Type type, string name, Type[] genericArgs, BindingFlags bindingAttrs); public static MethodInfo GetGenericMethod(this Type type, string name, Type[] genericArgs, Type[] types); public static MethodInfo GetGenericMethod(this Type type, string name, Type[] genericArgs, Type[] types, ParameterModifier[] modifiers); public static MethodInfo GetGenericMethod(this Type type, string name, Type[] genericArgs, BindingFlags bindingAttrs, Binder binder, Type[] types, ParameterModifier[] modifiers); public static MethodInfo GetGenericMethod(this Type type, string name, Type[] genericArgs, BindingFlags bindingAttrs, Binder binder, CallingConventions callConvention, Type[] types, ParameterModifier[] modifiers); public class Foo { public void Bar(object value) { ... } public void Bar(T value) { ... } } MethodInfo genericBar = typeof(Foo).GetGenericMethod(""Bar"", new Type[] { typeof(string) }, new Type[] { typeof(string) }); ``` " 645 area-System.IO Support long file names on Windows Now would seem to be the time to finally tackle the MAX_PATH nightmare. With the Win32FileSystem and other abstractions we can convert incoming paths to their long path equivalent if needed before handing off to the actual Win32 APIs. It appears that all of the APIs currently in use support the extended syntax (\?), with the exception of Get/SetDirectory. 649 area-System.Runtime String.Equals isn't intuitive and forces us to do the OrdinalIgnoreCase thing "We're doing a sample and we are FOREVER doing this ``` if (sort.Equals(""Title"", StringComparison.OrdinalIgnoreCase)) ``` I think it's time for an overload or an extension that ""does the right thing."" This is a 10 year old pattern and if the default is wrong (as we tell people to do this) then let's make a better method. Everyone has a version and extension that does this. " 652 area-Meta Remove dead code Many libraries and tests has some dead code. Please remove it per library. At minimum we should run FxCop (list might not be complete, following is just a list of folders under src/) - [ ] Microsoft.Win32.Primitives - [ ] Microsoft.Win32.Registry - [ ] System.Collections.Concurrent - [ ] System.Collections.Immutable - [ ] System.Collections.NonGeneric - [ ] System.Collections.Specialized - [ ] System.ComponentModel.Annotations - [ ] System.ComponentModel.Primitives - [ ] System.ComponentModel - [ ] System.Console - [ ] System.Diagnostics.FileVersionInfo - [ ] System.Diagnostics.Process - [ ] System.Globalization.Extensions - [ ] System.IO.FileSystem.DriveInfo - [ ] System.IO.FileSystem.Primitives - [ ] System.IO.FileSystem - [ ] System.IO.MemoryMappedFiles - [ ] System.IO.Pipes - [x] System.IO.UnmanagedMemoryStream (https://github.com/dotnet/corefx/pull/1080) - [ ] System.Linq.Parallel - [ ] System.Numerics.Vectors - [ ] System.Reflection.Metadata - [x] System.Resources.ResourceWriter - [ ] System.Runtime - [ ] System.Text.Encoding.CodePages - [ ] System.Text.RegularExpressions - [x] System.Threading.Tasks.Dataflow - [ ] System.Xml.XDocument - [ ] System.Xml.XPath.XDocument - [ ] System.Xml.XPath.XmlDocument - [ ] System.Xml.XPath - [ ] System.Xml.XmlDocument 657 area-System.Runtime System.Runtime tests fail on Win10, non en-US culture The code for the Single and Double.TryParse and ToString tests seem to assume OS < Win10 and CultureInfo = en-US. Here's a failure running on Win10 due to the same infinity issue as #136 ``` DoubleTests.TestToString [FAIL] Assert.Equal() Failure Γåô (pos 0) Expected: Infinity Actual: Γê₧ Γåæ (pos 0) Stack Trace: D:\Src\corefx\src\System.Runtime\tests\System\Double.cs(181,0): at DoubleTests.TestToString() DoubleTests.TestTryParse [FAIL] Assert.True() Failure Finished: System.Runtime.Tests Stack Trace: D:\Src\corefx\src\System.Runtime\tests\System\Double.cs(294,0): at DoubleTests.TestTryParse() SingleTests.TestToString [FAIL] Assert.Equal() Failure Γåô (pos 0) Expected: Infinity Actual: Γê₧ Γåæ (pos 0) Stack Trace: D:\Src\corefx\src\System.Runtime\tests\System\Single.cs(181,0): at SingleTests.TestToString() SingleTests.TestTryParse [FAIL] Assert.True() Failure Stack Trace: D:\Src\corefx\src\System.Runtime\tests\System\Single.cs(294,0): at SingleTests.TestTryParse() ``` I did not repro similar issues with non en-US culture, but I think from code inspection it won't work there either (even on Win7 or Win8)... 659 area-System.Runtime GCTest TotalMemory tests failing. ``` GCTests.TotalMemory [FAIL] Assert.True() Failure Stack Trace: d:\j\workspace\dotnet_corefx_prtest\Config\Release\OS\Windows_NT\Tests\false\label\windows\src\System.Runtime\tests\System\GC.cs(68,0): at GCTests.TotalMemory() ``` See: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_prtest/Config=Release,OS=Windows_NT,Tests=false,label=windows/91/console 668 area-System.Runtime Implement a generic Enum.Parse method Although the Enum.TryParse method (https://msdn.microsoft.com/en-us/library/system.enum.tryparse(v=vs.110).aspx) has two generic overloads, the Enum.Parse overloads (https://msdn.microsoft.com/en-us/library/System.Enum.Parse(v=vs.110).aspx) were added to the .NET Framework in version 1.0, before the introduction of generics. They create the need for rather redundant code, such as ``` Colors colorValue = (Colors) Enum.Parse(typeof(Colors), colorString); ``` which entails the need to add an additional cast, as well as instantiate a Type object that represents the enumeration type. It would be nice to add generic Parse overloads that correspond to the generic TryParse overloads: ``` public static TEnum Enum.Parse(String value); public static TEnum Enum.Parse(String value, Boolean ignoreCase); ``` Note that there's also another open issue on improving the performance of Enum.TryParse at https://github.com/dotnet/corefx/issues/594. 674 area-System.Diagnostics FileVersionInfo tests' native assemblies and exes do not have sources We should at least provide the sources for these native binaries so that we can tell what's going on if we hit issues. The next time we need to make a change to these tests, it would also be worth getting the native binaries to build as vcxprojs, relying on the VS2013 prerequisite for now (see [System.ServiceProcess.ServiceController.Tests](https://github.com/dotnet/corefx/tree/v1.0/src/System.ServiceProcess.ServiceController/tests/System.ServiceProcess.ServiceController.Tests) for an example). When we start to light up tests cross-platform, we will be forced to clean this up and use CMake or some other solution for native binaries. cc @weshaggard 679 area-Infrastructure Refactor build to run tests based on targets and dependencies rather than properties "The build system currently runs tests by setting the RunTestsForProject to execute after PrepareForRun. Because we don't want to run the tests after a Visual Studio build (because VS has its own mechanisms for running tests), since #508 the BuildingInsideVisualStudio property is used to disable the RunTestsForProject target. Making the build system aware of the context in which its running in order to skip a target feels odd in a build system with a robust target and dependency mechanism. It seems to me that it would be more idiomatic for the build to expose its targets in such a way so that each context can use the target system to control what targets are run. In other words, do not include the RunTestsForProject in the dependency tree of the default target for the sub-project files (which is what Visual Studio will execute), but do include it in the dependency tree of the default target for the top-level build file. In practice, I believe this would just mean changing the default target for each sub-project to be Compile instead of Build. This would also have the positive effect of making the build IDE-agnostic, which is important as .NET Core becomes truly cross-platform. What would the value of the ""BuildingInsideVisualStudio"" property be from within Xamarin Studio, or SharpDevelop, or Sublime using [OmniSharp](http://www.omnisharp.net/)? Xamarin Studio actually sets BuildingInVisualStudio to true in order to try and increase compatibility, which is obviously not a desirable state of affairs. Let's try to reduce the need for such hacks by keeping the dependencies between the build and the IDE clean (i.e. one-way). " 682 area-Meta Contract.Assert vs Debug.Assert Some of the CoreFX libraries use Contract.Assert. Others use Debug.Assert. We should be consistent, and if possible sweep the source to unify this. 683 area-System.Console Why having System.Console.dll? Why having a new DLL called `System.Console.dll`? `System.Console` class is already in `mscorlib.dll` since .NET Framework 1.0 684 area-System.Runtime Add non-throwing methods to Convert class "Convert class is the only method to convert boxed values to needed integer type. However, it throws exceptions, so it is too hard to implement behavior that matches simple explicit cast between integer types. As a solution - add TryXXX methods to Convert class. Additionally, it would be nice to have option to make Convert behave like explicit cast. For example, this is valid: ``` C# int a = -1; int b = (int)(uint)a; ``` But this is not: ``` C# int a = -1; int b = Convert.ToInt32(Convert.ToUInt32(a)); ``` This can be really useful when you process some data as ""just some bytes"" and you may get either signed or unsigned boxed value at input. As a possible solution - add overload that accepts second boolean argument and acts like an explicit integer cast. Example of this problems and required workaround can be seen in Enum class, in particular here: https://github.com/dotnet/coreclr/blob/master/src/mscorlib/src/System/Enum.cs#L207 " 686 area-Meta (Proposal) Coding Guidelines. "The coding guidelines should be alter to permit the following styles. which aids the readability of the code. - Align on the `:` of named parameters rather than lefthand edge of parmeter name ``` c# public static ImmutableArray GetAllMethodBodyDiagnostics(CSharpCompilation compilation, CancellationToken cancellationToken) { DiagnosticBag diagnostics = DiagnosticBag.GetInstance(); CompileMethodBodies( compilation: compilation, moduleBeingBuilt: null, generateDebugInfo: false, hasDeclarationErrors: false, filter: null, filterTree: null, filterSpanWithinTree: null, diagnostics: diagnostics, cancellationToken: cancellationToken ); DocumentationCommentCompiler.WriteDocumentationCommentXml(compilation, null, null, diagnostics, cancellationToken); compilation.ReportUnusedImports(diagnostics, cancellationToken); return diagnostics.ToReadOnlyAndFree(); } ``` vs ``` c# public static ImmutableArray GetAllMethodBodyDiagnostics(CSharpCompilation compilation, CancellationToken cancellationToken) { DiagnosticBag diagnostics = DiagnosticBag.GetInstance(); CompileMethodBodies( compilation: compilation, moduleBeingBuilt: null, generateDebugInfo: false, hasDeclarationErrors: false, filter: null, filterTree: null, filterSpanWithinTree: null, diagnostics: diagnostics, cancellationToken: cancellationToken); DocumentationCommentCompiler.WriteDocumentationCommentXml(compilation, null, null, diagnostics, cancellationToken); compilation.ReportUnusedImports(diagnostics, cancellationToken); * return diagnostics.ToReadOnlyAndFree(); } ``` - Simple IF If the If statement is small and has no else, and the code block is small enough. The combined width is less than the 80 character width limit. ``` c# if( arg0 == null ) throw new ArgumentException( nameof( arg0 ) ); if( arg0 == null ) { throw new ArgumentException( nameof( arg0 ) ); }; ``` vs ``` c# if( arg0 == null ) { throw new ArgumentException( nameof( arg0 ) ); } ``` Consider multiple guards statements you are approaching halve of the length of the screen, ever before you've reached the core of the method. - Simple case clause ``` c# select ( value ) { case 0: return ""Zero""; case 1: return ""One""; case 2: return ""Two""; // ... case 9: return ""Nine""; default: return ""Foo""; } ``` vs ``` c# select ( value ) { case 0: return ""Zero""; case 1: return ""One""; case 2: return ""Two""; // ... case 9: return ""Nine""; default: return ""Foo""; } ``` 15 LoC rather than 25 or 35 if you need to include the `break` ``` case 0: output = ""Zero""; break; ``` - 2 space indentation Consider the case of pattern-matching, each scope gets indented, 5 levels of indentation and you're already used a quarter of the permitted code width. - Method Parameter align vertically If the method signature is getting ""too"" wide, then consider vertical alignment. ``` c# internal static MethodSymbol DefineScriptEntryPoint ( CSharpCompilation compilation, PEModuleBuilder moduleBeingBuilt, TypeSymbol returnType, bool hasDeclarationErrors, DiagnosticBag diagnostics ) { ``` ``` c# internal static MethodSymbol DefineScriptEntryPoint(CSharpCompilation compilation, PEModuleBuilder moduleBeingBuilt, TypeSymbol returnType, bool hasDeclarationErrors, DiagnosticBag diagnostics) { ``` **NOTE** I not say that the existing code should be altered (it would still be valid according to the guidelines) but allow pull-requests to use some permitted alternatives forms. Yes someone could submit a pull-request that changes the exist to use the alternatives. Whether that pull-request would be accepted is a different question. " 692 area-System.Runtime Add a non-generic version of Enum.TryParse These are the methods in Enum class related to string parsing. ``` c# public static object Parse(Type enumType, string value); public static object Parse(Type enumType, string value, bool ignoreCase); public static bool TryParse(string value, out TEnum result) where TEnum : struct; public static bool TryParse(string value, bool ignoreCase, out TEnum result) where TEnum : struct; ``` Apparently Parse is exposed only in non-generic form, while TryParse is only exposed in generic form. This is somewhat inconsistent. It is simple enough to call a non-generic method in a generic method, but not the other way around. So Parse can be made generic using the following helper. But it is a lot harder to wrap TryParse into TryParse(Type enumType) without reflection calls. ``` c# public static TEnum Parse(string value, bool ignoreCase) { return Enum.Parse(typeof(TEnum), value, ignoreCase); } ``` The suggestion is to add a non-generic Enum.TryParse method, the API signature would look like ``` c# public static bool TryParse(Type enumType, string value, out object result) public static bool TryParse(Type enumType, string value, bool ignoreCase, out object result) ``` 693 area-System.Runtime Add Enum.SetFlag and Enum.RemoveFlag to make bitwise flagging easier To maintain multiple bool states in a class, instead of declaring many bool fields, an enum flag can be used to compact these states in to a single member to optimize for memory usage. ``` c# [Flags] enum DirtyFlags { X = 1, Y = 1 << 1, Z = 1 << 2 }; DirtyFlags dirtyFlags = DirtyFlags.X | DirtyFlags.Y | DirtyFlags.Z; ``` It is simple to test if a flag is set using `Enum.HasFlag(DirtyFlags.X)`, but the scenario is not complete. Without the ability to set and remove a flag, the following code needs to be maintained manually: ``` c# // Setting a flag dirtyFlags |= DirtyFlags.X; // Removing a flag, this is harder to remember :( dirtyFlags &= ^DirtyFlags.X; ``` The suggestion is to add 2 methods to Enum to help set and clear a flag. ``` c# public bool HasFlag(Enum flag); // This method already exists public bool SetFlag(Enum flag); public bool RemoveFlag(Enum flag); ``` These methods should be aggressively inlined since bitwise manipulations is very likely to be used in a performance sensitive scenario. 694 area-Meta State policy on mscorlib "I see a lot of confusion around mscorlib, since it's part of coreclr project, but it feels more like part of corefx. Some issues are open here (and sent to coreclr, like #594), some are open there and sent to here (like #684). Plus, there is some classes that exist in both repos with some differences like ConcurrentDictionary (for example of pull request open to one repo and sent to another see dotnet/coreclr#63). And even tests for runtime classes are in corefx (but runtime classes itself are in coreclr). There should be something like ""rule of a thumb"" to rule this out. " 699 area-System.Collections Implement ForEachAsync in System.Collections.Concurrent @stephentoub has some great blog posts about higher level operations that can be implemented all using public API that exists today. Some of these would be great to see in the library itself: http://blogs.msdn.com/b/pfxteam/archive/2012/03/05/10278165.aspx In particular ``` C# public static Task ForEachAsync(this IEnumerable source, Func body) { return Task.WhenAll( from item in source select Task.Run(() => body(item))); } ``` And ``` C# public static Task ForEachAsync(this IEnumerable source, int dop, Func body) { return Task.WhenAll( from partition in Partitioner.Create(source).GetPartitions(dop) select Task.Run(async delegate { using (partition) while (partition.MoveNext()) await body(partition.Current); })); } ``` 700 area-System.Runtime Proposal: System.Date type "Currently, trying to use System.DateTime to represent just calendar dates is overkill and is an easy way to introduce bugs. SQL Server supports a native date-only type, and having a date-only type in .NET to match would be very handy. But not only from SQL: accepting an MVC action method parameter from an HTML5 input type=""date"" field would be simplified with a native Date type. Examples: ``` C# Date d = new Date(2015, 2, 9); Date d2 = Date.UtcToday; Date d3 = Date.LocalToday; // helper methods for adding/subtracting days, months, years, or TimeSpan Date d4 = d.AddDays(7); Date d5 = d.Subtract(TimeSpan.FromYears(2)); string s = d.ToString(); //--> ""2015-02-09"" // properties: int y = d.Year; int m = d.Month; int day = d.Day; DayOfWeek weekday = d.DayOfWeek; int doy = d.DayOfYear; // convert from DateTime: Date d6 = DateTime.UtcNow.ToDate(); Date d7 = (Date)DateTime.UtcNow; // convert to DateTime: DateTime dt = d.ToDateTime(); DateTime dt2 = (DateTime)d; // equivalent to new DateTime(d.Year, d.Month, d.Day) ``` " 701 area-Infrastructure Add Contract Reference Asssemblies to the repo We need to enabling building our contract reference assemblies in the CoreFX repo. They are be needed for a number of reasons: 1) We need to build them to put them in our nuget packages. 2) We need them to produce our facades for our nuget packages. 3) People would like to see a reference for what is supported (see #147 as example) and for what we will be bringing online overtime. We will need to figure out the structure we want to provide them in and provide the tooling for editing them. A strawman for the structure would be something like: src\System.Runtime\src\Contract\4.0.0.0\System.Runtime.cs|csproj src\System.Runtime\src\Contract\4.0.10.0\System.Runtime.cs|csproj src\System.Runtime\src\Contract\4.0.20.0\System.Runtime.cs|csproj 703 area-System.Globalization Open-source System.Globalization unit tests Note: The implementation of this contract lives in mscorlib in the [coreclr](https://github.com/dotnet/coreclr) repository, but we are planning to put tests in this repo and test against the NuGet package produced by the coreclr repo. 704 area-System.IO Open-source System.IO unit tests Note: The implementation of this contract lives in mscorlib in the [coreclr](https://github.com/dotnet/coreclr) repository, but we are planning to put tests in this repo and test against the NuGet package produced by the coreclr repo. 705 area-System.Reflection Open-source System.Reflection unit tests Note: The implementation of this contract lives in mscorlib in the [coreclr](https://github.com/dotnet/coreclr) repository, but we are planning to put tests in this repo and test against the NuGet package produced by the coreclr repo. 707 area-System.Threading Open-source System.Threading unit tests Note: The implementation of this contract lives in mscorlib in the [coreclr](https://github.com/dotnet/coreclr) repository, but we are planning to put tests in this repo and test against the NuGet package produced by the coreclr repo. 708 area-System.Text Open-source System.Text.Encoding unit tests Note: The implementation of this contract lives in mscorlib in the [coreclr](https://github.com/dotnet/coreclr) repository, but we are planning to put tests in this repo and test against the NuGet package produced by the coreclr repo. 709 area-System.Diagnostics Write a test for PROCESS_QUERY_LIMITED functionality "Per #303 we have some debt here we need to get tested. It's not clear the best strategy here. Should this be an inner loop test? Does the test need to run elevated somehow or can we run at regular permissions and create a low integrity process? " 721 area-System.Net Registered I/O Extensions Expose Winsock's RIO Extensions to .NET. [Registered I/O is available](https://msdn.microsoft.com/en-gb/library/windows/desktop/ms740642%28v=vs.85%29.aspx) for desktop from Windows 8, Windows Server 2012, Windows Store apps from Server 2012 R2 and 8.1, and phone from Windows Phone 8 User code would want to pre-register a buffer, which would need to be pinned and go via [RIORegisterBuffer](https://msdn.microsoft.com/en-us/library/windows/desktop/hh437199%28v=vs.85%29.aspx) probably by passing through an internal buffer as per the [Websocket](http://referencesource.microsoft.com/#System/net/System/Net/WebSockets/WebSocket.cs,7dd0b6c7a0033d04)s to reduce data copies. Async sends and receives driven via ArraySegment over provided internal buffer. (Related [SO question](http://stackoverflow.com/questions/18419117/rio-registered-i-o-sockets-in-net)) 725 area-System.Console Augment terminfo search with first-letter-as-hex terminfo files are organized into subdirectories based on the first letter of the terminal name. On Linux, the letter itself is used as the directory name. On OS X, the hex value for the letter is used as the directory name. This commit updates the logic that searches for the terminfo database file to try both locations. Fixes #723 (the search portion of it, not the runtime asserts once the correct search path is used) 728 area-System.Text Open-source EncodingData tool This tool is used to generate [System.Text.Encoding.CodePages\src\System\Text\EncodingTable.Data.cs](https://github.com/dotnet/corefx/blob/master/src/System.Text.Encoding.CodePages/src/System/Text/EncodingTable.Data.cs). 735 area-Infrastructure NativeServiceTest fails to build with /p:OS=Unix "When running the following command from the root of my corefx repo: ``` build /t:rebuild /p:SkipTests=true /p:OS=Unix ``` I consistently get the following error towards the end of the build: ``` C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.Cpp.Platform.targets(57,5): error MSB8020: The build tools for v120 (Platform Toolset = 'v120') cannot be found. To build using the v120 build tools, please install v120 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting ""Retarget solution"". [\src\System.ServiceProcess.ServiceController\tests\NativeTestService\NativeTestService.vcxproj] ``` I have VS2015 CTP 5 installed. I don't have VS2013 installed. (We don't actually need this Windows-only test project to build when building with OS=Unix, if that's the source of the problem.) " 736 area-System.IO Use lseek instead of lseek64 in Unix I/O As part of this, changed interop definitions of off_t to be Int64 rather than IntPtr. For now we're assuming that we'll run either on 64-bit systems and/or where _FILE_OFFSET_BITS == 64. We can revisit this if that assumption fails. Fixes #733 738 area-System.Runtime Add more tests around Enum.TryParse 743 area-Infrastructure Feature: Need to create a solution for building localized binaries We need to come up with a solution to enable building localized binaries as well as taking contributions to localized content. 744 area-Infrastructure Top level bin\ output folder has become cluttered with overly generic names "Many of the files below are specific to a given library's tests, but there is no convention to identify them as such. e.g. 'Assembly1' is specific to FileVersionInfo tests and 'ModuleCore' is for Xml. We should either: 1. Rename all test assemblies to include a prefix matching their parent. 2. Create a hierarchical structure in bin\ I think 1 would be better as it would also make the build log easier to understand. I also question whether we even need so many test assemblies, especially in the XML case. `` D:\Src\corefx\bin\Debug>dir /b \* | findstr /v ""System"" | findstr /v ""Microsoft"" Assembly1 Axes.Tests BaseLibManaged Events.Tests MemoryMappedFile.Tests MemoryMappedViewAccessor.Tests MemoryMappedViewStream.Tests Misc.Tests ModuleCore NativeTestService ProcessTest_ConsoleApp Properties.Tests SDMSample.Tests Streaming.Tests TreeManipulation.Tests XDocument.Common XDocument.Test.ModuleCore XmlCoreTest XmlDiff xNodeBuilder.Tests xNodeReader.Tests XunitTraitsDiscoverers ``` ``` " 745 area-Infrastructure Many solutions fail to build in VS without a full build.cmd first The culprit in most cases is a missing project reference to XunitTraitsDiscoverers. VS needs the targets of project references to be in the scope of the current solution. While we could go and add this project everywhere, I think it will clutter the solutions. A better approach might be to use shared source files that we pull in the test targets/props. 746 area-Infrastructure The top-level src/ folder is cluttered with all of those .slns "IMHO, each library should have its sln file within its own folder. Instead of ``` src\ System.Xyz\ System.Xyz.sln ``` We should have ``` src\ System.Xyz\ System.Xyz.sln ``` I believe the reasons why we had the former originally are obsolete now. 1) We liked the idea of grouping related libraries together in a solution. Today, only System.Xml.\* does this. However, it will not really work long-term because we shouldn't have library-to-library project-to-project references as it messes with our package versioning story post-V1. 2) We had problems with redundant packages/ folders. I believe this was solved by the NuGet.config change that moved packages to a folder at the root outside of src " 754 area-System.Collections Simplify and modernize System.Collections tests The System.Collections tests were mechanically ported from existing test assets using a different test framework. Some of the tests also include misleading comments, and complication and obfuscation relating to constraints at the time they were written. 758 area-Infrastructure BuildPackages target fails because GetPackageVersion needs a version number "BuildPackages target is broken currently because GetPackageVersion requires more than we are currently setting in corefx repo right now. Luckily (:)) we had another bug that prevented this target from running on clean builds so not many people have hit it yet including our CI system. I'm fixing the other bug however I'm going to disable BuildPackages for now until we get this fixed. Task ""GetPackageVersion"" (TaskId:2) Task Parameter:NuSpecFile=D:\git\corefx\src\nuget\Microsoft.Bcl.Immutable.nuspec (TaskId:2) D:\git\corefx\packages\Microsoft.DotNet.BuildTools.1.0.25-prerelease-4\lib\packages.targets(62,5): error MSB4044: The ""GetPackageVersion"" task was not given a value for the required parameter ""DailyBuildNumber"". " 759 area-Infrastructure Incremental build tweaks Update Clean target to no longer delete PackagesDir, to address #529. Fixed an issue with the _RestoreBuildTools target where we sometimes get into a never ending running of that target in cases where your dir.props or pacakges.config is newer than the tools. This showed its head while I had edits to that file and was no longer cleaning my Packages directory. To address I always ensure the outputs are newer by touching them at the end of the target whenever it is ran. Also realized that our BuildPackages target was not being run any longer on clean builds and only on incremental builds. This showed that it is recently broken by the latest build tools update and I filed #758 for that but for now I'm disabling the building of the packaging. I also started importing dir.targets in build.proj so the tools were restored in time for them to be used in src\dir.proj even on clean builds. 765 area-Infrastructure Add daily build number to NuGet packages and fix package versioning. This is to address issue #758. This change adds BuildNumber.props containing the daily build number property that's consumed by the package build infrastructure. Updated package versions for Immutable and Metadata based on the values of the daily building packages. The values have a build number one greater than the last build. Deleted all nuspec files for non-shipping packages. 770 area-Infrastructure Make XunitTraitDiscoverers as a part of the custom Xunit. Inorder to skip tests and other xunit functionality, the projects require an extra project reference to XunitTraitDiscoverers. Making this a part of the netcore xunit will be good. It's basically an extension on Xunit functionalities, and we should merge these two projects. 774 area-Infrastructure Add System.Threading to packages.config in test projects "This would otherwise make the Mono mcs compiler fail with: ""error CS0518: The predefined type `System.Threading.Interlocked' is defined in an assembly that is not referenced."" It's the same change as in https://github.com/dotnet/corefx/pull/328 " 775 area-Infrastructure Use xunit netcore package in XunitTraitsDiscoverers XunitTraitsDiscoverers used the original xunit.abstractions and xunit.core packages instead of the netcore version. This causes a problem like https://github.com/dotnet/buildtools/pull/38 when compiling on Mono due to the unwanted xunit.runner.tdnet.dll 776 area-Infrastructure Fix condition in dir.targets The value passed to Exists() should be enclosed in single quotes according to https://msdn.microsoft.com/en-us/library/7szfhaft.aspx 777 area-Infrastructure Move solution files to the appropriate projects folders fixes #746 780 area-System.Collections Immutable collections test failure in CI An immutable collections test failed in a CI run: ``` System.Collections.Immutable -> d:\j\workspace\dotnet_corefx_windows_debug_prtest\bin\Windows_NT\Debug\System.Collections.Immutable\System.Collections.Immutable.dll System.Collections.Immutable.Tests -> d:\j\workspace\dotnet_corefx_windows_debug_prtest\bin\Windows_NT\Debug\System.Collections.Immutable.Tests\System.Collections.Immutable.Tests.dll xUnit.net console test runner (32-bit .NET Core) Copyright (C) 2014 Outercurve Foundation. Discovering: System.Collections.Immutable.Tests Discovered: System.Collections.Immutable.Tests Starting: System.Collections.Immutable.Tests System.Collections.Immutable.Test.ImmutableSortedSetTest.EmptyTest [FAIL] Assert.Same() Failure Expected: ImmutableSortedSet [] Actual: ImmutableSortedSet [] Stack Trace: d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Collections.Immutable\tests\ImmutableSortedSetTest.cs(354,0): at System.Collections.Immutable.Test.ImmutableSortedSetTest.EmptyTestHelper[T](IImmutableSet`1 emptySet, T value, IComparer`1 comparer) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Collections.Immutable\tests\ImmutableSortedSetTest.cs(82,0): at System.Collections.Immutable.Test.ImmutableSortedSetTest.EmptyTest() Finished: System.Collections.Immutable.Tests === TEST EXECUTION SUMMARY === System.Collections.Immutable.Tests Total: 591, Errors: 0, Failed: 1, Skipped: 0, Time: 1.839s ``` 783 area-Infrastructure Switch to running tests with new BuildAndTest target. An attempt to address #679 This change adds a BuildAndTest and Test target to our default dir.targets file and hooks RunTestsForProject off the Test target so that by default if you build an individual project from the command line or in an IDE like VS it will just do the build and not run the tests. If people want to run the tests then they need to execute the BuildAndTest target (or just the Test target). This prevents the need for us to use special conditions like BuildingInsideVisualStudio in our targets. I've also changed the DefaultTargets for our build.proj and dirs.proj files to be BuildAndTest so if you run the build for them it will stil do the build and test run. cc @eatdrinksleepcode @nguerrera 800 area-Infrastructure Enable code coverage on VS2015 and include more libraries Two issues: 1) Code coverage doesn't work with VS2015. 2) Several of our libraries are disabled from code coverage. For the first, the issue turned out to be a failure of the OpenCover code coverage tool to read the PDBs produced by Roslyn. The library on which the tool depends for PDB reading has been updated to support Roslyn, and a more recent version of OpenCover than the one we're currently using pulls in that updated version of the PDB reader. As such, the fix is just to upgrade which version of OpenCover we use. For the second, whatever reason caused us to disable code coverage on several projects appear to have since been addressed, either by us or in the OpenCover update. The fix then is to just re-enable code coverage on these libraries. The only library that remains disabled for code coverage is System.Diagnostics.Process.dll. It's either hanging or just taking a really, really long time (PLINQ takes about 10 minutes, but at least it completes). Needs more investigation. 810 area-System.Globalization Implement System.Globalization.Extensions on Unix The library has been refactored to have the Unix implementation stubbed out. It needs to be properly implemented. 811 area-System.Diagnostics Implement System.Diagnostics.FileVersionInfo on Unix The library has been refactored to have the Unix implementation stubbed out. It should be implemented to the extent that makes sense on Unix. 817 area-System.IO Implement System.IO.FileSystem.Watcher for Mac The Linux implementation uses inotify, which is Linux-specific. We'll need a different implementation for Mac, such as one based on FAM, kqueue, or FSEvents. 818 area-Infrastructure Update solutions/projects with OS-specific configs For projects that have OS-specific implementations, this change modifies their solution and project files to create OS-specific configurations. Rather than just having a Debug or Release option, in Visual Studio a developer can now select OS_Debug or OS_Release, e.g. for a project that has a Windows implementation and a Unix implementation, Windows_Debug, Windows_Release, Unix_Debug, and Unix_Release will all be available options. Changing the configuration will update the properties used to configure conditional includes, build outputs, etc. cc: @mmitche, @AArnott. Andrew, please let me know if there's a better way to achieve this. 820 area-Infrastructure Run compliance tool against Project K " " 821 area-Infrastructure Collect telemetry from test runs 822 area-Infrastructure Auto upload packages to nuget/myget 823 area-Infrastructure Strategy for test hooks/mocks that works everywhere 824 area-Infrastructure Contract Generation 825 area-Infrastructure Runtime Facade Generation 826 area-System.IO Open Source ClrCompression.dll 838 area-Infrastructure Dashboard: Caclulate product confidence from test and code coverage data. " " 839 area-Infrastructure enable perf and stress infrastructure 840 area-Infrastructure Align ToF and xunit reporting " " 841 area-Infrastructure Run OuterLoop tests on officially built binaries " " 846 area-System.Globalization Cross Platform System.Globalization "This a meta-workitem tracking a bunch of smaller stuff, now that we have multiple folks working on this stuff concurrently. - [x] TimeZone - [x] Encodings (#2774) - [x] Collation (#2769) - [x] Casing (#2772) - [x] Locales (#2770) - [x] Calendars (#2773) " 864 area-System.Reflection Open Source System.Reflection.Primitives Tests 875 area-Infrastructure OS-independent libraries should be built to an OS-independent folder Once #818 gets merged, we should refactor how our binaries are built to output paths. Libraries that have no direct OS-dependencies and do not have an OS-specific build should go into a neutral folder, and libraries that have OS-specific builds should go into an OS-specific folder. 880 area-Microsoft.CSharp Open Source Microsoft.CSharp.dll 882 area-System.Net Open Source Networking.dll 891 area-System.Net Open Source System.Net.Http.dll 892 area-System.Net Open Source System.Net.Http.WinHttpHandler.dll 893 area-System.Net Open Source System.Net.NameResolution.dll 894 area-System.Net Open Source System.Net.NetworkInformation.dll 895 area-System.Net Open Source System.Net.Primitives.dll 896 area-System.Net Open Source System.Net.Sockets.dll 897 area-System.Net Open Source System.Net.Utilities.dll 912 area-Infrastructure Update properties in dir.props to make them more consistent Used some msbuild string functions to help eliminate some of the duplications around OS/OSGroup/ConfigurationGroup. Explicitly started defining all the needed properties by build tools in a consistent way so the build upon each other. Restructured the output paths a little to eliminate many nested directories. Binaries: $(OS).$(Platform).$(Configuration)\$(MSBuildProjectName)\ Intermediates: obj\$(OS).$(Platform).$(Configuration)\$(MSBuildProjectName)\ Tests: tests\$(OS).$(Platform).$(Configuration)\$(MSBuildProjectName)\ 914 area-System.Text Write a test for GetGroup never returning Empty Group We merged #253 without a corresponding test. We need to write one. 915 area-Meta Remove uses of Console.WriteLine in the Unit Tests? A bunch of our unit tests do `Console.WriteLine` either on regular paths or on error cases. By my count there's bout 1600 hits for `Console.Write*` in our tests. This is a bit of an over-count because some of these are the Console tests themselves. I don't really like this very much. It feels like something we inherited when porting our legacy test assets. Is it worthwhile to go clean this up? I don't mind doing the work, just don't want to put in a bunch of effort if it doesn't stand a chance of getting merged. I'm sure @nguerrera will be interested in this and my understanding was @davkean had some guidance on how to author and port tests and I'd be interested in understanding how this aligns with his thinking. 921 area-System.ComponentModel Improve code coverage for System.ComponentModel.Annotations According to the current code coverage report, this library is only at 75%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 930 area-System.Numerics Improve code coverage for System.Numerics.Vectors According to the current code coverage report, this library is only at 72%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 931 area-System.Reflection Improve code coverage for System.Reflection.Metadata According to the current code coverage report, this library is only at 51%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 933 area-System.Text Improve code coverage for System.Text.Encoding.CodePages (66%) "According to the current code coverage report, this library is only at 66%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. " 934 area-System.Xml Improve code coverage for System.Xml.XPath According to the current code coverage report, this library is only at 51%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 935 area-System.Xml Improve code coverage for System.Xml.XPath.XDocument According to the current code coverage report, this library is only at 54.5%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 936 area-System.Xml Improve code coverage for System.Xml.XPath.XmlDocument According to the current code coverage report, this library is only at 15%! Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 937 area-System.Xml Improve code coverage for System.Xml.XmlDocument According to the current code coverage report, this library is only at 47%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 942 area-Infrastructure Failed CI build reported it completed successfully "PR #917 had a bad P/Invoke declaration that caused a TypeLoadException and a huge number of tests to fail. But even though the build lists a large number of test failures: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_prtest/138/#showFailuresLink the build still shows as green / successful, and GitHub reports ""all is well"". " 956 area-Meta VS automatically changing solution and csproj What are your thoughts about VS automatically changing stuff? I.e: https://github.com/dotnet/corefx/pull/954 VS updated version info. Also it removed new line character in the end of csproj (I removed this this change). Should we keep that kind of changes or revert them? I don't have strong opinion on that although I generally think that easier solution is always better: we should not manually change csproj & sln files so if VS wants to change it I would let it. 957 area-Infrastructure Dashboard: Hook up official build results. " " 961 area-Infrastructure Add IsInnerLoopTestProject and IsOuterLoopTestProject property support during the build "Add properties to disable selected projects from Inner- or Outer- loop run. By default all projects are Inner-loop. You can disable it from particular run by explicitly setting TestCategories property " 962 area-System.Xml Remove dependency on System.Xml.XPath.XmlDocument from System.Xml.XPath.XDocument tests "Currently the test is comparing two different implementations of XPath - one of them is assumed ""good"" as there are currently no known bugs to it. I will remove the dependency once it is no longer needed although it is helpful right now since XPath.XDocument has few bugs for investigation and it has proven to be helpful for me (there are few behavioral differences between implementations which we cannot remove now because of compat and that other implementation is helping figure out if this is known compat issue or a code/test defect). Please remove this dependency once the bugs are fixed. Related issues: https://github.com/dotnet/corefx/issues/641 https://github.com/dotnet/corefx/issues/284 https://github.com/dotnet/corefx/issues/17 " 963 area-Infrastructure Support Proxy servers for our nuget download task See PR #788 where we reverted the fix for #38 because there were some unhanded cases which caused a null reference if the DefaultWebProxy is null. We should add support again with proper checks in place. 970 area-System.Collections ReadOnlyObservableCollection.CollectionChanged should be made public "The `ReadOnlyObservableCollection.CollectionChanged` event is protected, which is flawed, it prevents you from being able to observe changes to it without casting to `INotifyCollectionChanged`. I suspect the original thinking was because the collection is ""read-only"" it never changes - this is not correct. _Read-only_ in .NET means that the consumer has a read-only view over a collection (which is why we project `IReadOnlyList` to `IVectorView`), not that the contents are immutable and never change. While we've made `protected` members `public` in the past (such as `Exception.HResult`), this event was marked `virtual`, which while binary compatible, would be a source-breaking change for those that overrode it. I suspect very little have overridden it, but we may want to consider adding a new event with a different name such as `Changed`. " 972 area-System.IO Fix for Issue #587 - Untangle FileStream's async implementation Fix for Issue #587 - Untangle FileStream's async implementation This removes the need for the old Begin\End methods in Win32FileStream by modifying the wrapper around the Overlapped (Win32FileStream.FileStreamAsyncResult) to use a TaskCompletionSource, thus allowing the core Read\Write methods to return Tasks directly. It also fixes a potential race condition where Begin\* may be registering the cancellation at the same time that End\* is trying to dispose it (if the user called the End\* method from the callback that they provided to the Begin\* method). 973 area-Meta .NET Core + Existing Platforms - What Can We Expect? I found myself watching a discussion between @davkean and @onovotny go down [[start here](https://twitter.com/davkean/status/568636866053812224)] to better understand where .NET Core sits relative to existing platforms. Because 140 characters is a terrible medium, let's keep this going here. I'm still looking at supporting Windows 7 users for a while at least, so I'm curious what options are available around continuing to build Portable Class Libraries (the plans here seem to be that referencing other NuGet packages is how you can access additional BCL functionality in the future - correct me if I'm wrong) over targeting .NET Core directly. What do we gain? What do we lose? And what expectations should we have in the short-term? 975 area-System.Runtime Fill System.Runtime.Extensions test gaps The initial merge of System.Runtime.Extensions tests (#802) included tests which we were running internally that were marked as covering the System.Runtime.Extensions contract. That set of tests does not cover all the types in the contract. We should add tests to cover the rest of this contract. 978 area-Infrastructure Dashboard: Enable User specific layouts and configs. "https://github.com/benjamine/jsondiffpatch can be used to greate user specific patch data to modify the global freeboard state. " 980 area-Infrastructure System.ObjectModel code coverage is failing === TEST EXECUTION SUMMARY === System.ObjectModel.Tests Total: 53, Errors: 0, Failed: 0, Skipped: 0, Time: 0.340s Committing... No results, this could be for a number of reasons. The most common reasons are: 1) missing PDBs for the assemblies that match the filter please review the output file and refer to the Usage guide (Usage.rtf) about filters. 2) the profiler may not be registered correctly, please refer to the Usage guide and the -register switch. 984 area-System.IO SerialPort Support on Windows Just wondering if SerialPort namespace will be included at some point? -------------------------------------------------------------------- ### Execution plan - [x] 1. @JeremyKuhne to seed the sources in CoreFX (from full .NET Framework) - add source code (sanitized, with licenses), the source won't build - [x] 2. Make the code build * CC: @JeremyKuhne @karelz on PRs * Note: We will not take any functional changes to the implementation, architecture, or API surface at this point unless they are absolutely necessary to make the code compile on .NET Core. Please give heads up on this issue as soon as you discover cases like that. - [x] 3. Add tests - tracking progress in #15752 - [x] @JeremyKuhne to find out if we can leverage full .NET Framework tests * We need to design the test environment - we need ability to test the code in CI (ideally without heavy modification of CI machines) * We need to set goal for code coverage If anyone is working on any step, please mention it & coordinate here to avoid duplicated effort. @karelz will co-assign the issue to you as well. 987 area-System.Reflection MetadataReader ReadFile fast path does not work with async file streams When a FileStream is opened with isAsync: true (.NET Core default!), the ReadFile P/Invoke code that we have in the metadata reader to read small files straight to native memory. ReadFile fails with invalid param error code because our handle was opened for overlapped IO. For now, I'm changing the code to be more defensive and always fall back to the slow file -> managed heap -> native heap code path. (It currently throws.) @ericstj: Can you help me figure out how to tweak FileStreamReadLightUp.cs to handle this? Also, @tmat has already mentioned to me that this code path will slow us down on Linux and Mac. Perhaps, this is the right opportunity to define a portable FileStream.Read overload that reads to unmanaged memory? 991 area-Infrastructure Enable code coverage report generation for individual projects If we run a subset of the projects for code coverage, we should still generate a report. 992 area-System.Numerics Consider saturated arithmetic functionality for Vector In the past, we've seen some requests for Vector to provide some means to do saturated arithmetic operations. Such operations could be useful for a variety of applications, like signal processing, image processing, etc. It makes sense to at least investigate what such functionality would look like, how it could be exposed, etc. Given that there are also SSE2+ instructions that expose some level of saturated arithmetic, I'm hopeful that we could also get some baseline hardware-instruction support for this. Work to do: - Define API surface exposed (which operations are supported) - Define which types support these operations (which integral types? all?) - Determine which operations could benefit from SIMD instructions - Create a simple working prototype (without hardware instructions) - Create formal API design speclet 993 area-System.Numerics Consider generic support for Shuffle and Shift operations on Vector Two useful operations that we have long-considered adding to Vector are shuffle and shift. These are pretty fundamental operations that low-level algorithms could very often take advantage of. One of the challenges was developing a generic but straightforward interface for utilizing such operations across all of the (integral) numeric types that Vector supports. Work to be done: - Define which operations are supported (shuffle at a minimum) - Determine which operations could benefit from SIMD instructions - These should be able to benefit greatly - Create a simple working prototype (without hardware instructions) - Create formal API design speclet and follow regular procedure 996 area-Infrastructure Dashboard: Add page to show test failures for specific area. " " 998 area-Infrastructure Dashboard: Make Bug Id link open in new window. " " 999 area-System.Runtime Add back Environment.Is64BitXXX Hi all. There are some methods that aren't yet implemented in Environment class. - Is64BitProcess - Is64BitOperatingSystem - OSVersion Will they be implemented? 1000 area-System.Diagnostics Several Dataflow ETW events aren't firing due to MethodAccessExceptions Several System.Threading.Tasks.Dataflow ETW events aren't firing on .NET Core, due to what looks like private reflection issues: MethodAccessExceptions are getting thrown and swallowed when firing these events. 1006 area-System.Net Add SmtpClient support Is there any plans to add `System.Net.Mail` into aspnetcore50? 1008 area-System.Numerics Faster and/or vectorized sin/cos In certain types of applications, the extra precision and correctness afforded by Math.Sin() and Math.Cos() are not worth the performance penalty they impose. These applications are often already using primarily single precision floats for computations, and thus have to round trip through double to make use of these methods. I propose adding a FastMath or Mathf or similar set of methods for lower precision high performance implementations of these methods. If it were not for the existence of System.Numerics.Vectors, I would advocate that these simply be an external library separate from the core framework, but since the vector math library ships Quaternion and Matrix support, I would vastly prefer that these types make use of the faster math operations for things like calculating rotations. My reasoning is that these types exist in the first place for people concerned with performance, and as it stands they're simply too slow compared to the alternatives for them to see much use. For a practical anecdote, I recently wrote an OpenGL draw call stress test to determine CPU overhead and how many draw calls I can push from my .NET app. By far the biggest consumer of frame time was Sin and Cos inside the quaternion rotation method used to rotate the cubes I was rendering. Using a fully managed implementation of SinCos based on the venerable CEPHES library cut that time roughly in half. This could be further extended by vectorizing with SIMD, provided that the Vector types support a few more intrinsics such as shuffle. I wouldn't mind doing this work if people think it would be useful. My main concern is that shipping System.Numerics.Vectors (quaternions and matrices) without these faster math routines will forever doom them to obscurity, since switching to reduced precision operations in the future would be a breaking change. 1009 area-Infrastructure Error message in System.ServiceProcess.ServiceController when building from VS2015 DevCmd When building using Developer Command from VS 2015 I get following error message: (build doesn't fail though - jut red message) ``` NativeTestService.cpp cl : Command line error D8027: cannot execute 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\c1xx.dll' [E:\oss\corefx\src\System.ServiceProcess.ServiceController\tests\NativeTestService\NativeTestService.vcxproj] ``` It doesn't repro on VS 2013 DevCmd. 1010 area-System.Numerics Add support for extracting a bit mask from a Vector SSE2/AVX supports extracting the MSB from each 8 byte,4 byte,2 byte or 1 byte segment (the `movemask_*` family). This can be extremely useful while performing algorithms that are data sensitive and use SSE registers like big buffers. A user could implement algorithms that work entirely in safe code that could benefit from avoiding the allocation and rigmarole of copying data out of a register to an array when you just want to test if a field is not empty. Admittedly, it is a downer as ARM does not have an intrinsic for this operation but there is workarounds that should perform better than copying everything in and out memory to test the bytes. http://stackoverflow.com/questions/11870910/sse-mm-movemask-epi8-equivalent-method-for-arm-neon 1016 area-System.Xml Update the XElement implementation in System.Xml.XDocument to include XmlSchemaProvider attribute once the packages are refreshed. 1019 area-Meta Consider adding Disposables [Reactive Extensions](https://github.com/Reactive-Extensions/Rx.NET/tree/master/Rx.NET/Source/System.Reactive.Core/Reactive/Disposables) and [System.Threading.Tasks.Dataflow](https://github.com/dotnet/corefx/blob/master/src/System.Threading.Tasks.Dataflow/src/Internal/ActionOnDispose.cs) are using their own internal Disposables. I don't know how many people have written similar APIs too. We also use our own disposables to run actions. This is very helpful for types implementing IObservable and types used within a `using`-statement. I think CoreFX is the right place to offer a small common API for disposables. What are your thoughts? 1021 area-Infrastructure Improving test coverage and relevance: investigate tools for code dependencies. Investigate tools that will show the dependencies between code and tests. This will allow us to only run the relevant set of tests related to new changes. 1022 area-Infrastructure MSBuild consistency in casing boolean properties "MSBuild properties are case sensitive. The ""false""/""true"" vs ""False""/""True"" What do we prefer? It looks like we don't have any standards with this and I saw various places in the targets which mix two different casings even for the same properties. I believe we should unify this Examples: src\System.Collections.Immutable\src\System.Collections.Immutable.csproj (lowercase): ``` true full false DEBUG;TRACE prompt 4 false System.Collections.Immutable.ruleset ``` and the same project in the bottom: ``` True True Strings.resx ``` src\dirs.targets: (Look at the IsTestProject property and CLSCompliant condition): https://github.com/dotnet/corefx/blob/master/src/dir.targets#L94 ``` True True False false ``` " 1033 area-System.Reflection Delimiter list in the type name grammer was incorrect "Duplicated ""*"" in the delimiter list, it should be ""\"". See Tokenizer.HandleEscapeSequence for more information. " 1034 area-Meta Update Developer Wiki with new section on Code Coverage The developer guide wiki (https://github.com/dotnet/corefx/wiki/Developer-Guide) should have a section on the code coverage system - including how to manually run the coverage build and generate a new report. 1037 area-System.Diagnostics Investigate why Assert went out of it's way to call MessageBox on a different thread There's some internal history on why this decision was made that @stephentoub and I both don't fully grok. I need to take the time and understand if this is still something we need to be doing and why it only mattered for CoreCLR and not desktop at the time. Once that's done, something like #1012 might be in order. 1039 area-Meta Why is DataTable/View/Set absent? I looked here: https://github.com/dotnet/corefx-progress and saw DataTable/View/Set isn't ported to .NET Core. IMHO this is a mistake, as DataTable and friends are a convenient way to handle untyped data at runtime, and is e.g. often used when fetching data through stored procedures as it's easy to e.g. fetch a dataset with the result of a proc which returns 3 cursors on e.g. Oracle. I fail to see why this cornerstone of many people's data-access code is absent in a framework that targets servers. 1044 area-Infrastructure Microsoft.DotNet.TestHost.1.0.3-prerelease only targets 32-bit "Running tests on Window 7 64-bit fails due to `CoreRun.exe` being 64-bit and the TestHost dlls being 32-bit. When setting the 32-bit flag on `xunit.console.netcore.exe` with `corflags.exe` and invoking it directly (iow without `corerun.exe`), the tests work again. ``` > dumpbin /headers CoreRun.exe | find ""machine"" 8664 machine (x64) > dumpbin /headers api-ms-win-core-console-l1-1-0.dll | find ""machine"" 14C machine (x86) ``` " 1045 area-System.Xml XmlReader.ReadAsync goes into infinite recursion "**Before fixing the bug please make sure the tests for Xml.ReaderWriter were already added or you can add your own tests - make sure you test old and new behavior. Any breaking changes need to be additionally reviewed.** _This is a modified copy of internal bug, may contain fragments of discussions I not necessarily written myself_ Repro: ``` csharp XmlReader xmlReader = XmlReader.Create(""http://www.example.com/example.xml"", new XmlReaderSettings{Async = true}); await xmlReader.ReadAsync(); ``` > I investigated this issue and believe the root cause is that the support for this scenario (UriString) is not yet implemented, that causes the termination statement ""laterInitParam = null"" inside FinishInitUriStringAsync() never gets a chance to execute and thus the infinite recursion. Open issues: 1) XmlReader.Create, XElement.Load, XDocument.Load overloads with ""string uri"" are not supported 2) Characters that are not supported in the current encoding are not automatically encoded as entities Code change which caused break: ``` csharp private Task FinishInitAsync() { switch (laterInitParam.initType) { // code below was removed when removing Uri support //case InitInputType.UriString: // return FinishInitUriStringAsync(); //<= This is never executed and thus leads to the infinite recursion. case InitInputType.Stream: return FinishInitStreamAsync(); case InitInputType.TextReader: return FinishInitTextReaderAsync(); default: //should never hit here Debug.Assert(false, ""Invalid InitInputType""); return AsyncHelper.DoneTask; } } ``` Tasks: - [ ] Determine if we want to support Uri or not - [ ] Fix the issue " 1050 area-System.Reflection Metadata reader CanReadFromSameMemoryMappedPEReaderInParallel test failed in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_prtest/257/ ``` System.Reflection.Metadata.Tests.MetadataReaderTests.CanReadFromSameMemoryMappedPEReaderInParallel [FAIL] System.AggregateException : One or more errors occurred. ---- System.BadImageFormatException : Invalid COR20 header signature. ---- System.BadImageFormatException : Invalid COR20 header signature. Stack Trace: Discovered: System.ComponentModel.Primitives.Tests at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure) at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally) --- End of stack trace from previous location where exception was thrown --- at System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection exceptions, CancellationToken cancelToken, Exception otherException) at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally) at System.Threading.Tasks.Parallel.For(Int32 fromInclusive, Int32 toExclusive, Action`1 body) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\tests\Metadata\MetadataReaderTests.cs(2457,0): at System.Reflection.Metadata.Tests.MetadataReaderTests.CanReadFromSameMemoryMappedPEReaderInParallel() ----- Inner Stack Trace #1 (System.BadImageFormatException) ----- d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\MetadataReader.cs(173,0): at System.Reflection.Metadata.MetadataReader.ReadMetadataHeader(BlobReader& memReader) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\MetadataReader.cs(97,0): at System.Reflection.Metadata.MetadataReader..ctor(Byte* metadata, Int32 length, MetadataReaderOptions options, MetadataStringDecoder utf8Decoder) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\PEReaderExtensions.cs(71,0): at System.Reflection.Metadata.PEReaderExtensions.GetMetadataReader(PEReader peReader, MetadataReaderOptions options, MetadataStringDecoder utf8Decoder) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\PEReaderExtensions.cs(47,0): at System.Reflection.Metadata.PEReaderExtensions.GetMetadataReader(PEReader peReader) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\tests\Metadata\MetadataReaderTests.cs(2457,0): at System.Reflection.Metadata.Tests.MetadataReaderTests.<>c__DisplayClass2c.b__2a(Int32 _) at System.Threading.Tasks.Parallel.<>c__DisplayClass9`1.b__12(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Threading.Tasks.Parallel.<>c__DisplayClass9`1.b__12(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute() ----- Inner Stack Trace #2 (System.BadImageFormatException) ----- d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\MetadataReader.cs(173,0): at System.Reflection.Metadata.MetadataReader.ReadMetadataHeader(BlobReader& memReader) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\MetadataReader.cs(97,0): at System.Reflection.Metadata.MetadataReader..ctor(Byte* metadata, Int32 length, MetadataReaderOptions options, MetadataStringDecoder utf8Decoder) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\PEReaderExtensions.cs(71,0): at System.Reflection.Metadata.PEReaderExtensions.GetMetadataReader(PEReader peReader, MetadataReaderOptions options, MetadataStringDecoder utf8Decoder) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\src\System\Reflection\Metadata\PEReaderExtensions.cs(47,0): at System.Reflection.Metadata.PEReaderExtensions.GetMetadataReader(PEReader peReader) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Reflection.Metadata\tests\Metadata\MetadataReaderTests.cs(2457,0): at System.Reflection.Metadata.Tests.MetadataReaderTests.<>c__DisplayClass2c.b__2a(Int32 _) at System.Threading.Tasks.Parallel.<>c__DisplayClass9`1.b__12(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Threading.Tasks.Parallel.<>c__DisplayClass9`1.b__12(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute() ``` 1051 area-System.Text RegexGroup should provide a Name #### Background Regex group can have names assigned to them (e.g. `(?.+)`). Those names can be used in an indexer of `RegexGroupCollection`, however the `RegexGroup` objects themselves do not provide a `Name` property. The names can be retrieved using `regex.GetGroupNames`, but it is not immediately obvious -- and I'm not sure it is even possible if static `Regex` method was used for matching. To prove it is not obvious, see this [Regex: get the name of captured groups in C#](http://stackoverflow.com/questions/1381097/regex-get-the-name-of-captured-groups-in-c-sharp) which at the moment had 48 votes on question, and 66 on answer. There are a few similar questions as well. #### API Design ``` csharp public class RegexGroup : ... { // ... public string Name { get; } } ``` Note that for numbered groups this should return group index as string, for consistency with `Regex.GetGroupNames`. #### Compatibility At the moment I do not see any potential compatibility issues with adding a property. 1052 area-System.IO Initial commit of System.IO.FileSystem tests (File, Directory, FI, DI) Initial commit of ported tests for the System.IO.FileSystem contract. Includes tests for File, Directory, FileInfo, and DirectoryInfo. A few tests have been rewritten in the xunit style, but the majority of these tests have been ported from ToF, which were ported from old tree. When porting, I modified or disabled tests to make sure every test only writes in its working directory. I verified this with PerfView. I also replaced hard-coded paths with calls to Path.GetRandomFileName() to prevent multiple runs from interfering with each other, and multiple tests running in parallel from interfering with each other. Any test that used a delay to wait for the state of the file system to stabilize was marked with OuterLoopAttribute. On my machine, all of the tests including OuterLoop take between 10s and 15s to run. Just inner loop tests take between 1s and 2s to run. Lots of these tests are candidates for rewriting, and Eric introduced a useful base class for tests when he rewrote the FileStream tests. Checking in these ported assets will give us a good idea of what the coverage is, and act as a baseline to compare the rewritten tests against. 1053 area-System.ComponentModel Make DisplayNameAttribute available in .NET Core This was originally going to be a request to move `DisplayNameAttribute` into Core, but the problem seems a little bigger than that. Currently, `System.dll` houses approximately a couple hundred types spread across four dozen different namespaces, but not everything in it is listed on the [Open Source Progress table](https://github.com/dotnet/corefx-progress/blob/master/src-diff/README.md) (although a select few namespaces like `System.Security.Cryptography.X509Certificates` _are_ there). This still makes some namespaces (for example `System.ComponentModel`) feel incomplete because of how they're spread across different assemblies. The previously mentioned `X509Certificates` namespace is already being tracked with all its types from `System.dll`, so other namespaces should follow suit, right? I was porting a project of mine over to see the ASP.NET vNext templates in action, but the lack of `DisplayNameAttribute` is literally the only thing preventing me from building it (even if I tell it to not target .NET Core) without commenting those attributes out. And if .NET Core's philosophy is to only use what you need, why should I be forced to import some monolithic DLL with hundreds of types that I don't need just for the one that I _do_ need, because that type is not in the DLL named after its containing namespace? ¯\(°_o)/¯ 1054 area-Infrastructure Add NuGet.org and MyGet.org latest build feeds to readme.md The CoreFX NuGet.org and MyGet.org latest build / package feeds should be added to our README.md. See https://github.com/xunit/xunit as an example. 1058 area-Infrastructure Add Windows 7 coverage to Jenkins / Outerloop runs Currently build.cmd no longer passes on Windows 7 machines (see https://github.com/dotnet/corefx/issues/1044). We should look into adding Win7 coverage to Jenkins to ensure we don't regress building or running tests on this platform. 1063 area-Meta Append guidelines with how PR feedback should be addressed "Pull request guidelines should contain something like ""rule of a thumb"" which rule how PR feedback should be addressed or how merge conflicts should be resolved. As example consider #972. Commits like `Merge branch 'master' into Issue587` (aaa5434) clutter history and should be avoided. `git rebase` should be used for updating branch with fix. Other commits in this PR could be squashed. As another example consider PR #1003. It's initial commit of the library so it's OK to address feedback in a separate commit. However in this PR feedback addressed twice and the second could be amended to the first. Conclusion: 1. Merge conflicts should be resolved through rebasing. 2. PR feedback should be in a separate commit if it contains important changes. Changes like formatting or renaming variable introduced in PR are not important and could be addressed through amending/rebasing. " 1068 area-System.Reflection Allow ISignatureTypeProvider to be a struct The `SignatureDecoder` takes the provider as a naked interface. This makes it inefficient to bake it by a struct because it will cause it being boxed each time it's passed. However, it's very desirable to make the provider a struct because it allows the caller to provide call site specific information. For example, in order for implementers to reference the generic type parameters of the method or type, the `ISignatureTypeProvider` needs to have access to the containing type / containing method. To avoid the boxing operation, the methods simply needs an additional contraint like so: ``` csharp public static TType DecodeType(Handle handle, TProvider provider) where TProvider : ISignatureTypeProvider ``` Unfortunately, this causes some noise as C# is no longer able to infer the type arguments, so each call site must now qualify the type arguments. 1071 area-System.Reflection Merge master to dev/metadata In particular, the changes from PR #1026 are needed to address the CI failure in #1068. There is no code review required as these commits have already been reviewed and there are no merge conflicts. 1072 area-System.Collections Remove source control bindings VS will keep prompting developers what to do with those. Eventually, they will show up as a diff, so it's best to commit those as a one-off. 1073 area-Infrastructure Rename all test assemblies to include a library prefix Fixes #744 Open question: what to do with - BaseLibManaged - ModuleCore - XDocument.Common - XDocument.Test.ModuleCore - XmlCoreTest - XmlDiff 1079 area-System.Security Add leaveOpen constructor overload for CryptoStream "A CryptoStream always closes the wrapped stream when it is disposed. This not ideal when using a MemoryStream or when another action needs to be taken on the result. The workarounds are manually calling `FlushFinalBlock()` and then getting the MemoryStream bytes before disposing the CryptoStream, or using a custom stream that ignores the dispose call from CryptoStream. Neither are ideal and can be avoided with a ""leaveOpen"" option. Implementation would be the same as DeflateStream, on `Dispose()` check if leaveOpen is true and don't close the internal stream. ``` public CryptoStream(Stream stream, ICryptoTransform transform, CryptoStreamMode mode, bool leaveOpen) ``` I'm happy to do the change when System.Security.Cryptography.Encryption arrives. " 1081 area-System.Diagnostics Proposal: Add new property for getting process path "## Rationale and Usage In order to get process path you need to use `someProcess.MainModule.FileName` property. However it suffers from a couple of issue: - Accessing MainModule of 64 bit process from a 32 bit one throws an exception - You can't access MainModule of elevated process from a non elevated one As you can see many developers [encounter](http://stackoverflow.com/q/8431298/239438) [these](http://stackoverflow.com/q/9501771/239438) issues. I suggest adding a new property which returns the path to executable of the Process. To overcome the issues that I described we can use `QueryFullProcessImageName` together with `PROCESS_QUERY_LIMITED_INFORMATION` access right. ## Api Proposal ```csharp namespace System.Diagnostics { public class Process : Component { public string MainModuleFilePath { get; } } } ``` ## Open Questions Should the new property be called `MainModuleFilePath` or something else? From my own experience most of people use `Path` when asking question on stackoverflow but @Priya91 suggested calling it `MainModuleFilePath` as it is returning Path of the `MainModule` property. ## Cross Platform Support To implement this property on other platforms on Linux we can use `readlink` to read the value of `""/proc/%d/exe""` symlink and on OS X we can use `proc_pidpath` which returns process path given process pid." 1085 area-System.Reflection MetadataReader: (THandle)(Handle)default(THandle) should round-trip for all handles For most handles default(THandle) is the canonical nil value that can convert to the weakly-typed Handle and back, but for some such as StringHandle and NamespaceHandle default(THandle) is an invalid value that becomes a nil module handle on conversion to Handle and cannot be round-tripped back to THandle. A fix is proposed in #1075. Filing a bug to track it so that I can close a duplicate internal bug against it. 1086 area-System.IO Add API to get actual file casing in path "Per @jasonwilliams200OK in dotnet/coreclr#390 Based on this answer: http://stackoverflow.com/a/81493/1712065 (further redirected from: http://stackoverflow.com/a/326153). Please add the ability to retrieve path with actual case via `FileInfo` and `DirectoryInfo` classes. The candidate member being `FullPath` and `Name`. Perhaps there is some sophisticated way of getting it from win32 file system API, but [that](http://stackoverflow.com/a/81493/1712065) seems to be a working solution. Expected: ``` c# using System.IO.FileInfo; // .. // actual path was: // C:\\SharePoint\\scripts\\MasterDeployment.ps1 FileInfo info = new FileInfo(""c:\\sharepoint\\scriPts\\maSTerdeploymnet.PS1""); Console.WriteLine(info.FullPath); // C:\\SharePoint\\scripts\\MasterDeployment.ps1 Console.WriteLine(info.Name); // MasterDeployment.ps1 ``` Actual result: ``` c# using System.IO.FileInfo; // .. // actual path was: // C:\\SharePoint\\scripts\\MasterDeployment.ps1 FileInfo info = new FileInfo(""c:\\sharepoint\\scriPts\\maSTerdeploymnet.PS1""); Console.WriteLine(info.FullPath); // c:\\sharepoint\\scriPts\\maSTerdeploymnet.PS1 Console.WriteLine(info.Name); // maSTerdeploymnet.PS1 ``` " 1088 area-System.Xml Remove link to a document that no longer exists There is a link to an internal SharePoint site that no longer exits, so it's unclear what the content was but the gist seems to be contained in the code comment. 1091 area-Infrastructure Projects fail to load in VS without having built on command line first "Since the new resources targets were added, projects cannot be opened in VS from a clean enlistment with no packages having been restored yet. **Repro steps** 1. Delete corefx/packages/ folder 2. Open a solution in VS ``` C:\src\corefx\src\System.Reflection.Metadata\src\System.Reflection.Metadata.csproj : error : The imported project ""C:\src\corefx\packages\Microsoft.DotNet.BuildTools.1.0.25-prerelease-00015 \lib\resources.targets"" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk. C:\src\corefx\src\dir.targets ``` " 1093 area-Infrastructure Create an optimization story for open-source compiled assemblies We need to design and develop the optimization story for our open-source corefx libraries. 1099 area-Infrastructure Build the exclusion list of CT errors against Project K " " 1100 area-System.Diagnostics StartInfo may return incorrect information for *running* processes. ProcessStartInfo.EnvironmentVariables gives an incorrect value for a _running_ process. We fetch environment variables via the GetEnviornmentVariables which gives the information of the current process and not of the _running_ process. Desktop also has the same behavior. 1101 area-System.Reflection Fix CustomAttributeNamedArgument to return the correct value There is a bug where `CustomAttributeNamedArgument` returns the type instead of the value. Strong typing can't help with `object`, can it? @nguerrera 1102 area-System.Numerics Further BigInteger test refactoring - Completes the removal of Console.WriteLine() from BigInteger tests and general refactoring to better use XUnit Asserts. (Progress against https://github.com/dotnet/corefx/issues/915.) - Generalizes the switch statement in Driver.cs's Worker class. - Removes duplicate code in StackCalculator.cs by referencing the identical implementations in MyBigInt.cs. - Removes duplicate implementations of Print(), GetRandomByteArray(), and IsZero() by moving them to MyBigInt.cs. - Enables a set of existing but unused tests: - RunDivRemTests - RunIsEvenTests - RunIsOneTests - ModPowValidSmallNumbers - RunMultiplyTests (If there is history for why theses tests were disabled/never enabled, please let me know, but it looks like a set of refactoring mistakes.) The first five commits are basically VS regex find-replace with the exception of the [changes to Driver.cs](https://github.com/dotnet/corefx/commit/2b7616cc33009422a662f34891406c755eecb4a6) mentioned above. The last four are more general refactoring to reduce copy/past function implementations. Please let me know how (or if) you want these commits squashed before merging. 1103 area-Infrastructure Coverage runs failing since ProcessTests changes @pallavit, since the recent ProcessTests cleanup changes got committed, all of our code coverage runs have hung. They've all hit an AV exception, followed by what appears to be OpenCover hanging during its subsequent processing: ``` xUnit.net console test runner (32-bit .NET Core) Copyright (C) 2014 Outercurve Foundation. Discovering: System.Diagnostics.Process.Tests Discovered: System.Diagnostics.Process.Tests Starting: System.Diagnostics.Process.Tests Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. at System.AppDomain.SetupDomain(Boolean allowRedirects, String path, String configFile, String[] propertyNames, String[] propertyValues) System.Diagnostics.ProcessTests.ProcessTest.Process_HasExited [FAIL] Process_HasExited001 failed Stack Trace: d:\j\workspace\dotnet_corefx_coverage_windows\src\System.Diagnostics.Process\tests\System.Diagnostics.Process.Tests\ProcessTest.cs(178,0): at System.Diagnostics.ProcessTests.ProcessTest.Process_HasExited() Finished: System.Diagnostics.Process.Tests === TEST EXECUTION SUMMARY === System.Diagnostics.Process.Tests Total: 34, Errors: 0, Failed: 1, Skipped: 0, Time: 120.241s Committing... Visited Classes 23 of 35 (65.71) Visited Methods 186 of 357 (52.1) Visited Points 1570 of 2998 (52.37) Visited Branches 515 of 1161 (44.36) ==== Alternative Results (includes all methods including those without corresponding source) ==== Alternative Visited Classes 27 of 40 (67.5) Alternative Visited Methods 193 of 415 (46.51) ``` @sawilde, FYI. 1105 area-Infrastructure CI code coverage runs failing due to invalid UploadCoverallsData "Our CI code coverage runs get all the way to the end and then fail with this error: ``` d:\j\workspace\dotnet_corefx_coverage_windows\build.proj(36,7): error MSB4113: Specified condition ""'$(UploadCoverallsData)'"" evaluates to """" instead of a boolean. ``` " 1107 area-Infrastructure Official builds should place tags on what was built "We're eventually going to want to be able to answer the question of ""what code was in build [whatever]?"". It seems the answer to this in git is tags. As an example, https://github.com/torvalds/linux, click on the branches dropdown and change to the tabs tab. Supporting documentation: http://git-scm.com/book/tr/v2/Git-Basics-Tagging " 1110 area-Infrastructure corefx helloworld Hi, I want to use corefx components and coreclr in my command line program. I started with corefxlab helloworld and I tried to add corefx from nuget manager. I was able to find corefx components but when I tried to install, I get the following error message. How do I overcome this? ``` Could not install package 'System.Globalization 4.0.10-beta-22231'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.5', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author. ``` Also, how do I install my private coreCLR and corefx components to my commandline program? I am using VS 2015 CTP6. Thanks 1118 area-Meta Use TimeSpan everywhere we use an int for seconds, milliseconds, and timeouts "In a large software project, it's easy to mess up units for time. IE, is that Int32 seconds or milliseconds? Or maybe minutes? I've been fixing my team's source code to use the unit-agnostic TimeSpan class wherever possible. However the .NET Framework is not complete in its adoption of TimeSpan. Specifically, we don't have a Process.WaitForExit overload that takes a TimeSpan, only an Int32 for the timeout. I suggest someone look through all .NET Framework API's for Int32 parameters containing ""second"", ""millisecond"", ""ms"", ""timeout"" (and perhaps ""time"") and see if there is a parallel TimeSpan-based overload. If not, please fix that. " 1121 area-System.Linq Allow customization of LINQ to XML string content. This issues comes about as a result of the following question on Stack Overflow: http://stackoverflow.com/questions/28947090/culture-and-linq-to-xml There appears to be no way to customize how a custom object is serialized to XML. The main problem this causes in my code is related to culture, but this could apply to other areas too. Looking at the code, it seems that there is no customization point for an object passed as content to LINQ to XML. Is this correct? Would it be a good idea to provide one? 1128 area-Microsoft.CSharp Add Microsoft.CSharp source 1129 area-System.Numerics More support for Math functions in System.Numerics.Vectors In particular I need Math.Exp support, similar to how there is support for Math.Sqrt via the Vector.SquareRoot function. It should be trivial to add support for Math.Exp and in fact I've done this with my local copy of System.Numerics.Vectors using SquareRoot as a guiding example. My application now uses the new Vector.Exp function instead of me having to use the CopyTo function to unpack my Vector to a double[] array, perform Math.Exp on each element of the array and then pack up into a new Vector. May I check the changes in? 1131 area-System.Linq Add System.Linq Source and Tests 1132 area-System.Security Need EncryptedXml and supporting types in Core framework This will likely depend on a large pile of other work, but I'd like to raise the issue. In porting applications to use CoreCLR, I have run into the need for the [EncryptedXml class](https://msdn.microsoft.com/en-us/library/system.security.cryptography.xml.encryptedxml%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396) more than once. From looking around, there are various 3rd party encryption libraries that exist but this adds a cost to porting from 1st-party .NET types. Enabling EncryptedXml in the Core framework will also enable encrypted configuration files in ASP.NET 5 Core profile as well... currently the user must create their own implementation of an encrypted configuration on this platform. 1133 area-System.Linq Add System.Linq.Queryable Source and Tests 1136 area-System.Collections Failure in System.Collections.Specialized.Tests.GetItemObjTests in CI MESSAGE: Error, returned wrong item for lowercase key +++++++++++++++++++ STACK TRACE: at System.Collections.Specialized.Tests.GetItemObjTests.Test01() in d:\j\workspace\dotnet_corefx_windows_release_prtest\src\System.Collections.Specialized\tests\HybridDictionary\GetItemObjTests.cs:line 362 1149 area-System.Linq Refactor System.Linq Following @stephentoub comment here: https://github.com/dotnet/corefx/issues/1143#issuecomment-81683842 I am filing this issue to track doing some refactoring in System.Linq. Currently, everything is in one file which has about 3.2k lines and around 20 classes and interfaces. It is good if this can be refactored to make the code easier to follow. 1151 area-System.Numerics New API for single-precision math ### Rationale The .NET framework does not currently provide scalar single-precision floating-point support for many of the trigonometric, logarithmic, and other common mathematical functions. Single-precision floating-point support should be provided for these mathematical functions in order to better interop with high-performance, scientific, and multimedia-based applications where single-precision floating-points are the only implementation required and/or used. Since adding these methods to the existing `System.Math` class would break backwards compatibility, they should be provided through a separate class that provides the same and/or similar functionality. Providing these new APIs would: - improve code readability (especially in the non-hardware backed implementations of methods in the `System.Numerics.Vector` structs) - improve code performance in areas where the single-precision implementation is all that is required - provide better interoperability with high-performance, scientific, and multimedia-based applications where data structures and existing code may be dependent on single-precision implementations of such methods. - provide a more advanced infrastructure for mathematics in the CoreCLR ### Proposed API The proposed API here provides feature-parity with the existing double-precision math functions provided by the framework. #### System.BitConverter ``` C# public static class BitConverter { public static float Int32BitsToSingle(int); public static int SingleToInt32Bits(float); } ``` #### System.Single ``` C# public static partial class MathF { public const float E = 2.71828183f; public const float PI = 3.14159265f; // Trigonometric Functions public static float Acos(float); public static float Asin(float); public static float Atan(float); public static float Atan2(float, float); public static float Cos(float); public static float Sin(float); public static float Tan(float); // Hyperbolic Functions public static float Cosh(float); public static float Sinh(float); public static float Tanh(float); // Exponential Functions public static float Exp(float); // Logarithmic Functions public static float Log(float); public static float Log(float, float); public static float Log10(float); // Power Functions public static float Pow(float, float); public static float Sqrt(float); // Rounding Functions public static float Ceiling(float); public static float Floor(float); public static float Round(float); public static float Round(float, int); public static float Round(float, int, MidpointRounding); public static float Round(float, MidpointRounding); public static float Truncate(float); // Manipulation Functions public static int Sign(float); // Minimum, Maximum, and Difference Functions public static float Max(float, float); public static float Min(float, float); // Other Functions public static float Abs(float); public static float IEEERemainder(float, float); } ``` ### Example Usage Currently to calculate the Tangent for each member of the `System.Numerics.Vector4` struct, you currently have to call the double-precision version of the method and cast to the result to a single-precision value: ``` C# public static Vector4 Acos(Vector4 value) { return new Vector4((float)(Math.Acos(value.X)), (float)(Math.Acos(value.Y)), (float)(Math.Acos(value.Z)), (float)(Math.Acos(value.W))); } ``` With the proposed changes, this would now be simplified to the following: ``` C# public static Vector4 Acos(Vector4 value) { return new Vector4(Mathf.Acos(value.X), Mathf.Acos(value.Y), Mathf.Acos(value.Z), Mathf.Acos(value.W)); } ``` The `System.Numerics` library itself is filled with similar examples as are various bits of code in the CoreFX and CoreCLR repositories. ### Perf Numbers All performance tests are implemented as follows: - 100,000 iterations are executed - The time of all iterations are aggregated to compute the `Total Time` - The time of all iterations are averaged to compute the `Average Time` - A single iteration executes some simple operation, using the function under test, 5000 times The execution time below is the `Total Time` for all 100,000 iterations, measured in seconds. **Hardware: Desktop w/ 3.7GHz Quad-Core A10-7850K (AMD) and 16GB RAM** | Function | Improvment | Execution Time - Double | Execution Time - Single | | :-: | :-: | :-: | :-: | | Abs | 0.199243555% | 0.63752649s | 0.63625626s | | Acos | 12.30220910% | 11.5265412s | 10.1085220s | | Asin | 18.66801808% | 11.9472425s | 9.71692911s | | Atan | 21.10350002% | 10.9964683s | 8.67582861s | | Atan2 | 20.51327307% | 24.3328097s | 19.3413540s | | Ceiling | 12.91487191% | 1.87116459s | 1.62950608s | | Cos | 5.026665542% | 7.19916547s | 6.83728750s | | Cosh | 16.46166555% | 13.5416170s | 11.3124413s | | Exp | 33.67586387% | 6.65578424s | 4.41439140s | | Floor | 10.39208688% | 1.74655247s | 1.56504922s | | Log | 19.81117664% | 6.42244806s | 5.15008553s | | Log10 | 18.40605725% | 6.75118866s | 5.50856101s | | Pow | 47.85595440% | 31.8820155s | 16.6245727s | | Round | 0.976398142% | 4.22620632s | 4.18494172s | | Sin | 15.49539339% | 5.98022268s | 5.05356365s | | Sinh | 17.96609899% | 14.6242270s | 11.9968239s | | Sqrt | 4.676516651% | 2.51281945s | 2.39530703s | | Tan | 30.33470555% | 9.07290178s | 6.32066374s | | Tanh | 0.108182099% | 8.12724112s | 8.11844890s | I believe some extra perf will be squeezed out when the intrinsics (such as `CORINFO_INTRINSIC_Sqrt`) are properly implemented in the VM layer for single-precision values. Without such functionality, it falls back to the double-precision functionality (extra precision, reduced performance) for certain calls. ### Pull Request There is a sample pull request covering these changes available: dotnet/coreclr#5492 ### Additional Details This will require several changes in the CoreCLR as well to support the new APIs via FCALLs and Intrinsics. 1153 area-System.Console Bring back Console.ReadKey() For reference, see issue #311 and the related aspnet/Hosting#140. Console.ReadKey() can solve both issues (thought explicit support for CancelKeyPress is nice as that can handle different keyboard configurations etc). ReadKey() is a helpful API for many console app scenarios and is quite difficult for users to implement in a non-Windows-specific way if they want to get the same functionality. Please consider bringing it back. 1155 area-System.Linq Ordering of ParallelEnumerable.GroupJoin and Join "This feels strange or bug-ish to me, but I'm not seeing anything either way in documentation. The following: ```c# ParallelQuery left = Enumerable.Range(1, 4).AsParallel().AsOrdered(); ParallelQuery right = Enumerable.Range(8, 32).AsParallel().AsOrdered(); foreach (Tuple> p in left.GroupJoin(right, x => x, y => y / 8, (k, g) => Tuple.Create(k, g))) { Console.Write(p.Item1.ToString() + "":""); foreach (int i in p.Item2) { Console.Write(i.ToString() + "",""); } Console.WriteLine(); } Console.Read(); ``` ... has output like this: ![groupjoin](https://cloud.githubusercontent.com/assets/5569508/6683955/d5efffe4-cccf-11e4-8958-4dfde0993f1d.jpg) The first grouping is out of order, whereas I'm expecting even the grouped elements to be returned in order (which is what the non-parallel version does). Strangely, of the multiple times I've run it, only the first run has ever been out of order. Using `OrderBy` on the right query tends to result in other runs being disordered.... `Join` has similar problems to `GroupJoin`: ```c# ParallelQuery left = Enumerable.Range(1, 4).AsParallel().AsOrdered(); ParallelQuery right = Enumerable.Range(8, 32).AsParallel().AsOrdered(); int prev = 0; foreach (Tuple p in left.Join(right, x => x, y => y / 8, (k, g) => Tuple.Create(k, g))) { if (prev != p.Item1) { Console.WriteLine(); Console.Write(p.Item1.ToString() + "":""); prev = p.Item1; } Console.Write(p.Item2.ToString() + "",""); } Console.Read(); ``` ... is producing output that looks like this: ![join](https://cloud.githubusercontent.com/assets/5569508/6684267/ee062632-ccd2-11e4-8363-679f7ed8dd0b.png) ...where every run is disordered. This is the behavior of the current consumer release as well. So is this ""intended"" behavior, and the documentation should be updated to warn people about the change from the sequential version (which explicitly states the ordering of the child elements is preserved), or is this an actual bug? @stephentoub , @svick , any thoughts? " 1165 area-System.IO FileSystemWatcher_InternalBufferSize_File test failed in CI FileSystemWatcher_4000_Tests.FileSystemWatcher_InternalBufferSize_File [FAIL] Should not observe a error event Stack Trace: d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.IO.FileSystem.Watcher\tests\Utility\Utility.cs(100,0): at Utility.ExpectNoEvent(WaitHandle eventOccured, String eventName, Int32 timeout) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.IO.FileSystem.Watcher\tests\FileSystemWatcher.InternalBufferSize.cs(53,0): at FileSystemWatcher_4000_Tests.FileSystemWatcher_InternalBufferSize_File() Finished: System.IO.FileSystem.Watcher.Tests 1168 area-System.Numerics Vector Shuffling Operations "As mentioned in #993, a lot of useful SIMD code relies on shuffling data around within registers. There are a whole bunch of instructions available to do this, so we'd like to expose a coherent subset of these via Vector. ## Rationale and Usage Simple stream processing, such as the operations already supported on Vector, are entirely parallel and thus don't benefit from shuffling operations. However, lots of common uses of SIMD do require it, such as doing geometric vector operations like matrix multiplies and vector cross products. These are key scenarios for certain users, such as game developers. If rearranging elements is required for an algorithm and fast hardware support is not provided, the cost of loading and storing to and from SIMD registers might exceed any benefits gained from using SIMD in the first place. When looking at existing libraries that wrap SIMD functionality at a low level, it's apparent that there are a few main use cases of shuffle: - Permuting the elements of two different vectors into a result - ""Swizzling"" the elements within a single vector - ""Splatting"" or ""broadcasting"" one element to all others within a single vector Rather than expose one single shuffle method directly, I think it makes sense to go with this slightly higher level functionality such that the API doesn't become too tied to a particular architecture's instruction set. ## Proposed API New methods on the Vector struct: ``` csharp // pick elements from a and b to place into result internal static unsafe Vector Permute2 (Vector a, Vector b, int s0, int s1); internal static unsafe Vector Permute4 (Vector a, Vector b, int s0, int s1, int s2, int s3); // rearrange elements within a internal static unsafe Vector Swizzle2 (Vector a, int s0, int s1); internal static unsafe Vector Swizzle4 (Vector a, int s0, int s1, int s2, int s3); // rearrange bytes in a internal static unsafe Vector ShuffleBytes (Vector a, Vector mask); // broadcast specified element to all other elements in a internal static unsafe Vector Splat (Vector a, int element); ``` New methods on Vector class that are simple wrappers around the Vector methods. ``` csharp public static Vector Permute (Vector a, Vector b, int s0, int s1); public static Vector Permute (Vector a, Vector b, int s0, int s1); public static Vector Permute (Vector a, Vector b, int s0, int s1); public static Vector Permute (Vector a, Vector b, int s0, int s1, int s2, int s3); public static Vector Permute (Vector a, Vector b, int s0, int s1, int s2, int s3); public static Vector Permute (Vector a, Vector b, int s0, int s1, int s2, int s3); public static Vector Swizzle (Vector a, int s0, int s1); public static Vector Swizzle (Vector a, int s0, int s1); public static Vector Swizzle (Vector a, int s0, int s1); public static Vector Swizzle (Vector a, int s0, int s1, int s2, int s3); public static Vector Swizzle (Vector a, int s0, int s1, int s2, int s3); public static Vector Swizzle (Vector a, int s0, int s1, int s2, int s3); public static Vector ShuffleBytes (Vector a, Vector mask); public static Vector Splat (Vector a, int element); ``` ## Details and Questions - There are a lot of tradeoffs when picking method signatures for shuffle. For example, instead of taking many selector parameters, we could instead have one combined integer mask. However, pretty much every single caller will then implement their own helper method to do the bitwise combination of the mask, so all callsites get more complicated. Additionally, we want the JIT to discover and optimize the cases where the parameters are JIT-time constants. - I chose for the selector parameters to be ints instead of, say, bytes. I'm not sure if there's a compelling reason to choose one over the other. In either case, we'd have to define what happens when the parameters are out of range. Since the intrinsics are going to bitwise combine them, in the interests of speed it'd be good if they could say that are not range checked. - We could also have a Permute8 and Swizzle8 and extend the methods to shorts and ushorts. The parameter lists get kind of unwieldy... - ShuffleBytes is an interesting case. pshufb is a very useful instruction to expose, but it's an SSSE3 instruction. It may be worth being more explicit about which features are supported by the hardware via some set of caps flags instead of the single boolean IsHardwareAccelerated we have now. - Shuffle intrinsically breaks the abstraction of Vector of not needing to know the underlying register size. Honestly, I think that's an unworkable abstraction in the long term; it's too difficult to write usage code that can work with arbitrarily sized vectors, so it might be worth exploring having specifically sized Vector128, Vector256, etc. This is clearly a discussion to be had in a separate issue though. " 1170 area-System.Collections IEnumerator.Current does not throw after hitting the end of the collection. This issue is in the `List.Enumerator` class. When the collection is enumerated to then end and then modified, accessing `IEnumerator.Current` returns `default(T)` rather than throwing. According to the [documentation](https://msdn.microsoft.com/en-us/library/vstudio/system.collections.ienumerator.current%28v=vs.100%29.aspx) it should throw. Simple Test case: ``` CSharp var list = new List{0,1}; var enumerator = ((IEnumerable)list).GetEnumerator(); while (enumerator.MoveNext()){} list.Add(2); Assert.Throws(() => enumerator.Current); ``` This test fails currently with `enumerator.Current` returning 0. 1171 area-System.Collections Add tests for KeyedCollection. Issue #867 "This change adds tests for `KeyedCollection` ported from internal test assets, bringing test coverage from 0 to 96% This also includes tests for `IList`, `ICollection`, and `IEnumerable` that can be used across all tests involving collections. I will be bringing over tests for `IList`, `ICollection`, and `IEnumerable` in a future PR. " 1178 area-System.Linq Add System.Linq.Expressions source and tests. 1182 area-System.Linq System.Linq performance improvement suggestions "With Linq-to-Objects it is quite common practice to perform a series of transformations, and then materialize sequence to a concrete collection type by calling _ToArray(), ToList(), ToDictionary()_. These operations would work much faster if they knew the number of elements in the sequence. Currently System.Linq.Enumerable has special treatment for _ICollection_ interface only. I suppose that additional support for _IReadOnlyCollection_ can improve performance in some cases, because through it we can figure out the total number of elements. Another problem with System.Linq is that in many cases the information about the number of elements is lost. One of the most common example: ``` C# List source = ...; List transformed = source.Select(o => o + 1).ToList(); ``` Obviously _Select_ does not change the number of elements in the sequence, but _ToList_ method cannot take advantage of that. Information is lost. In this particular scenario, it would be great for _Select_ to return some _SelectIterator_ instance, which implements _IReadOnlyCollection_, and thereby passes the number of elements to the subsequent methods. **Steps to measure performance gain**: 1. Find in _System.Linq.Tests\Performance_ the test for the method you've changed; 2. Uncomment `[Fact]` attribute above that method; 3. Build test project by Visual Studio in Release mode; 4. Go to the folder with tests binaries: `bin\tests\Windows_NT.AnyCPU.Release\System.Linq.Tests\aspnetcore50\`; 5. Open command prompt (cmd.exe or PowerShell); 6. Run command: `CoreRun.exe xunit.console.netcore.exe System.Linq.Tests.dll -parallel none -trait ""Perf=true""`; 7. Wait for results that will be printed right in the command window; 8. Don't forget to run tests with different collection sizes. This can be done by varying _elementCount_ argument of _Measure_ method. **Casting to `IReadOnlyCollection`** is a slow operation so it is not a good idea to check if this interface implemented. The performance drop will be most noticeable on the small collections. **Tasks**: - [ ] `Select`: add iterators for `List`, `Array[T]`, `ICollection`; - [ ] `ToArray`, `ToDictionary`: add special support for `ICollection` (and, very carefully, for `IReadOnlyCollection`) to get the initial capacity; - [ ] `ToList` (???): add special support for `IReadOnlyCollection` to get the initial capacity (separated as it will affect _System.Collections.Generic_); - [ ] `OrderBy(Descending)/ThanBy(Descending)`: implement special iterator for `ICollection` to propagate `Count`; - [ ] `Cast`, `Reverse`: add iterators for `ICollection` to propagate `Count`; - [ ] `Range`, `Repeat`: add an iterator that implements `ICollection`; - [ ] `Skip`, `Take`: add an iterator that handle `ICollection`; - [x] Add performance tests for _System.Linq_ " 1187 area-System.Collections Improve the exception message of ToDictionary when keys are duplicated "`Enumerable.ToDictionary` makes it easy to turn an array into a dictionary. But when the keys are duplicated in the source array, it just throws an `ArgumentException` that says **An entry with the same key already exists**. It is hard to know what keys are duplicated. To improve the exception message without adding extra overhead, the exception message can be modified to **$""An entry with the key { key } already exists""**. `ToDictionary` internally calls `Dictionary.Add()`, so the improvement made to Dictionary.Add benefits all code that calls the method. " 1198 area-System.Linq Improve code coverage for System.Linq.Expressions According to the current code coverage report, this library is only at ~40%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 1199 area-Serialization Improve code coverage for System.Xml.XmlSerializer According to the current code coverage report, this library is only at ~42%. Our general goal is closer to 90%, though this will depend on the library. We should investigate to see whether the current coverage level is appropriate, and if not, improve our test suites to provide better coverage. 1218 area-System.Collections Provide ConcurrentDictionary the ability to set initial capacity with DefaultConcurrencyLevel "It's absurdly difficult to create a ConcurrentDictionary with an initial capacity. This forces you to set the arcane DefaultConcurrencyLevel integer. I absolutely want to defer that to the platform. This leads to using this horrific code: ``` c# (int)typeof(ConcurrentDictionary<,>).MakeGenericType(typeof(int), typeof(int)) .GetProperty(""DefaultConcurrencyLevel"", BindingFlags.Static | BindingFlags.NonPublic) .GetValue(new ConcurrentDictionary()); ``` So either add a constructor that allows this. Add magic number support that providing < 1 results in this, or just make DefaultConcurrencyLevel be a public property. " 1220 area-Infrastructure Need a mechanism to be able to run dangerous Outerloop tests (that require machines to be recycled) Of the System.IO.FileSystem tests ported over, there were a couple that modified the current directory. Some set the current directory directly, some used the CurrentDirectoryContext helper class. Most of these instances were testing the behavior of APIs that acted on relative paths. For example, CreateDirectory_DotDotAsPath_WhenCurrentDirectoryIsRoot_DoesNotThrow() in CreateDirectory.cs does what its title says. This is a specific instance of the general problem of testing global process state. 1221 area-System.IO System.IO.FileSystem tests need a mechanism to test mounting drives System.IO.FileSystem tests need a way to test machine-wide state, specifically mounting network drives. 1227 area-System.Linq Performance tests for System.Linq (issue #1182) This PR introduces the performance tests for System.Linq as a part of work on the issue #1182. 1235 area-Serialization Add some tests for serialization Add/update serialization tests: DCS_BaseClassAndDerivedClassWithSameProperty Xml_StringAsRoot 1241 area-System.Linq Adding more tests to System.Linq. This time I copied some files from System.Linq.Parallel, because, looking at them, they seem to fit very well with System.Linq after removing the AsParallel() keyword. If I get a positive feedback about this from the community, then I will go ahead with copying the rest of files from System.Linq (of course on the condition that they fit) and hopefully this will improve the code coverage of System.Linq considerably. #1143 1243 area-Infrastructure Build should disallow OS versions (and other parameters) that aren't supported We should have an allowable OS list and disallow parameters not matching the list. This avoids issues where someone sets OS=Mac vs OS=OSX. 1244 area-System.Runtime Add overloads to string trimming "The proposal is to add new extension methods / overloads for string trimming, As you know, right now, it's only possible to trim individual characters from a string, but I would like to trim suffixes & prefixes. An example usage: `""MyTest"".TrimEnd(""Test"") == ""My"" == true` I feel like they should've be there. It seems kind of weird to me, as I've implemented these by myself in the past few times, and am sure there are quite few people who miss this & would find it useful: http://stackoverflow.com/questions/7170909/trim-string-from-end-of-string-in-net-why-is-this-missing http://stackoverflow.com/questions/4101539/c-sharp-removing-strings-from-end-of-string http://stackoverflow.com/questions/5284591/how-to-remove-a-suffix-from-end-of-string http://stackoverflow.com/questions/4335878/c-sharp-trimstart-with-string-parameter Now the following statement is not true, but if it would be, it would describe how I am feeling: ""ooh hey, we offer you string replace method which can replace individual chars, but not the whole strings. It's not too hard to craft your own one, give it a try!"" The following applies to `TrimEnd` / `TrimStart`, but the overloads would be 1:1, so I will discuss only `TrimEnd`. **First overload:** `public string TrimEnd(string suffix)` Behaviour: trim the suffix, case-sensitive, and only once. The comparision is done the same way, as it would be for `string.Replace`. `""MyTestTest"".TrimEnd(""Test"") == ""MyTest"" == true` **Second overload:** `public string TrimEnd(string suffix, StringComparison comparison)` Works as the first one, but allows you to explictly tell how to compare. `""MyTESTtest"".TrimEnd(""test"", StringComparison.InvariantCultureIgnoreCase) == ""MyTEST"" == true` **Third overload(s):** I am not sure if these are needed, but I wanted to throw this out here anyway: ``` public string TrimEnd(string suffix, bool trimRepeatedly) public string TrimEnd(string suffix, bool trimRepeatedly StringComparison comparison) ""MyTestTEST"".TrimEnd(""test"", true, StringComparison.InvariantCultureIgnoreCase) == ""My"" == true ``` This proposal has nothing to do with string.Trim(), as it would be ambigious. `""tetet"".Trim(""tet"") == ???` Namespace: _System_ Type: _System.String_ Assembly: _System.Runtime.dll_ I'd be willing to work on this :3 " 1247 area-Serialization Serialization tests fail ``` DataContractSerializerTests.DCS_ContainsLinkedList [FAIL] System.Exception : Test failed for input : SerializationTypes.ContainsLinkedList Stack Trace: d:\Workspace\corefx\src\System.Runtime.Serialization.Xml\tests\DataContractSerializer.cs(1717,0): at DataContractSerializerTests.SerializeAndDeserialize[T](T value, String baseline, DataContractSerializerSettings settings, Func`1 serializerFactory) d:\Workspace\corefx\src\System.Runtime.Serialization.Xml\tests\DataContractSerializer.cs(1113,0): at DataContractSerializerTests.DCS_ContainsLinkedList() Starting: System.Xml.XPath.XmlDocument.Tests Finished: System.Runtime.Serialization.Xml.Tests === TEST EXECUTION SUMMARY === System.Runtime.Serialization.Xml.Tests Total: 97, Errors: 0, Failed: 1, Skipped: 0, Time: 1,906s ``` # ``` DataContractJsonSerializerTests.DCJS_ContainsLinkedList [FAIL] System.FormatException : Input string was not in a correct format. Stack Trace: at System.Text.StringBuilder.AppendFormatHelper(IFormatProvider provider, String format, ParamsArray args) at System.String.FormatHelper(IFormatProvider provider, String format, ParamsArray args) at System.String.Format(IFormatProvider provider, String format, Object[] args) at System.Diagnostics.Debug.WriteLine(String format, Object[] args) d:\Workspace\corefx\src\Common\tests\System\Console.cs(12,0): at System.Console.WriteLine(String message, Object[] args) d:\Workspace\corefx\src\System.Runtime.Serialization.Json\tests\DataContractJsonSerializer.cs(1291,0): at DataContractJsonSerializerTests.SerializeAndDeserialize[T](T value, String baseline, DataContractJsonSerializerSettings settings, Func`1 serializerFactory, Boolean skipStringCompare) d:\Workspace\corefx\src\System.Runtime.Serialization.Json\tests\DataContractJsonSerializer.cs(1047,0): at DataContractJsonSerializerTests.DCJS_ContainsLinkedList() xUnit.net console test runner (32-bit .NET Core) Copyright (C) 2014 Outercurve Foundation. Discovering: System.Xml.XPath.Tests Finished: System.Runtime.Serialization.Json.Tests === TEST EXECUTION SUMMARY === System.Runtime.Serialization.Json.Tests Total: 74, Errors: 0, Failed: 1, Skipped: 0, Time: 1,477s ``` /cc @khdang 1254 area-System.Linq Enumerable.Single and Enumerable.SingleOrDefault overloaded implementations are inconsistent There is an inconsistency between the implementations of [Enumerable.Single()](https://github.com/dotnet/corefx/blob/master/src/System.Linq/src/System/Linq/Enumerable.cs#L1260) and [Enumerable.Single(predicate)](https://github.com/dotnet/corefx/blob/master/src/System.Linq/src/System/Linq/Enumerable.cs#L1284). Similarly there is another similar inconsistency between the implementations of [Enumerable.SingleOrDefault()](https://github.com/dotnet/corefx/blob/master/src/System.Linq/src/System/Linq/Enumerable.cs#L1306) and [Enumerable.SingleOrDefault(predicate)](https://github.com/dotnet/corefx/blob/master/src/System.Linq/src/System/Linq/Enumerable.cs#L1330) The overloads which accept a predicate enumerate the whole sequence before eventually throwing, even if more than one matching elements were already found: For consistency and performance reasons, it may be a good idea to make [Enumerable.Single(predicate)](https://github.com/dotnet/corefx/blob/master/src/System.Linq/src/System/Linq/Enumerable.cs#L1284) and [Enumerable.SingleOrDefault(predicate)](https://github.com/dotnet/corefx/blob/master/src/System.Linq/src/System/Linq/Enumerable.cs#L1330) methods able to fail fast and to throw an `InvalidOperationException` as soon as a second matching element is found. If that seems like a reasonable change, I would gladly submit a pull request. 1255 area-Infrastructure Create Win7 CI build We currently have a number of folks that are still working on Win7 which we support and we keep breaking things from time to time. We need to get a CI build/test for Win7. 1256 area-Serialization Fix issue with serialization of XmlElement "From #1199 The following code runs on .net 4.5 but does not run on corefx. There is an exception {""To be XML serializable, types which inherit from IEnumerable must have an implementation of Add(System.Object) at all levels of their inheritance hierarchy. System.Xml.XmlElement does not implement Add(System.Object).""} System.Exception {System.InvalidOperationException} ``` csharp public class Program { public void Main(string[] args) { XmlSerializer serializer = new XmlSerializer(typeof(Product)); string text = @"" Iphone smartphone ""; StringReader reader = new StringReader(text); Product product = (Product)serializer.Deserialize(reader); Console.ReadLine(); } } public class Product { [XmlAnyElement] public XmlElement[] AllElements; } ``` " 1271 area-System.Runtime Add overload for case insensitive string Replace "# Introduction `String` does not provide `Replace` method for case-insensitive replacements, but only for case-sensitive replacements. # Rationale and usage The idea here is to make programmers life easier & move closer to API consistency. It is fairly common scenario where you have user inserted data that has some kind of place-holders (tokens), that later have to be replaced by some templating system. Current solutions in the internet involve using regex class for this trivial task. ```c# var userTemplateData = ""Hello there. Time now is %currenttime%""; var evaluatedData = Regex.Replace( userTemplateData, ""%CurrentTime%"", DateTime.Now.ToString(), RegexOptions.IgnoreCase); // evaluatedData = ""Hello there. Time now is 01/04/2015 14:37""; ``` This approach is horrible: 1) not readable 2) often times not correct, as pattern has to be run through `Regex.Escape` before. 3) most likely slow(er) After the necessary change is added, we can do: ```c# var evaluatedData = userTemplateData.Replace( ""%CurrentTime%"", DateTime.Now.ToString(), StringComparison.InvariantCultureIgnoreCase); ``` How common would the usage of this be? I would **speculate** that every programmer in their career will hit the lack of case-insensitive replacement in framework at least once. I can't say much more, expect provide some StackOverflow links below in Details section. I did quick search through the code-base I am new to, and found around ~30 case-insensitive `Regex.Replace` usages. # Proposed API ```c# public sealed class String : ... { .... // these two methods are provided for framework consistency // trying to follow the StartsWith/EndsWith public string Replace(string oldValue, string newValue, StringComparison comparison); public string Replace(string oldValue, string newValue, bool ignoreCase, CultureInfo culture); public string Replace(char oldValue, char newValue, CompareOptions options); .... } ``` # Details Namespace: **System** Type: **System.String** Assembly: **System.Runtime.dll** I'd love to see the current `string.Replace` [CoreClr implementation](https://github.com/dotnet/coreclr/blob/cbf46fb0b6a0b209ed1caf4a680910b383e68cba/src/classlibnative/bcltype/stringnative.cpp#L835) pulled out from CoreClr, and implemented 100% in the `String.cs`: This would help to: - avoid code-duplication - make it more transparent (programmers can take look at the implementation) - unit-test a little less, as they share both common functionality for `Replace(..., StringComparison.Ordinal);` Alternatively, we could also modify the existing `CoreClr` implementation to deal with `StringComparison`. Or, just rewrite everything from zero. # Links to explore http://stackoverflow.com/questions/244531/is-there-an-alternative-to-string-replace-that-is-case-insensitive http://stackoverflow.com/questions/6275980/string-replace-by-ignoring-case http://weblogs.asp.net/jongalloway/71188 # Updates - Edited this description to make it more of a **speclet**, based on the discussion below and the proposed API Review process. - Feedback to @tarekgh. Added my opinion about how often this functionality would be used I'd be willing to work on this :3 " 1272 area-System.Collections Make System.Collections.Immutable [Serializable] Please make collections from namespace System.Collections.Immutable serializable if it is possible - mark them with `[Serializable]` attribute and do other related changes. There can be many cases when this would be useful. Thanks. 1285 area-System.Reflection Misc additions supporting Portable PDB reader 1297 area-System.Linq Unit Tests for Enumerable.Select Set of basic tests for: ``` IEnumerable Enumerable.Select( this IEnumerable source, Func selector) ``` 1. Verify `ArgumentNullException`is thrown when either `source` or `selector` is null. 2. Verify that deferred execution works (for `T[]`, `List` and `IEnumerable`) 3. Verify that returned collection contains expected items (for `T[]`, `List` and `IEnumerable`) 4. Verify that `GetEnumerator` called twice returns two different instances of enumerator. Tests are intended to improve code coverage around `Select` method before applying performance improvements to it (https://github.com/dotnet/corefx/pull/1282). It's also connected to https://github.com/dotnet/corefx/pull/1241 and https://github.com/dotnet/corefx/issues/1143. 1298 area-System.IO System.IO.Compression test asset and native dependency proposal "System.IO.Compression provides an API to compress and decompress files. It supports the ZIP and gzip archive formats and the Deflate compression algorithm. **_Testing**_ System.IO.Compression has existing test assets that fit three categories: 1. Round trip data through compression and verify it remains intact 2. Extract and verify data compressed through other compatible compression libraries 3. Verify behavior when attempting to decompress invalid or edge case archives The existing assets include archives, as well as decompressed binary files both for use as source material, and an oracle to verify decompression. Altogether, the binary assets weigh in at about 40MB. **Problem:** these test assets need to exist at test run time. Placing them in our corefx git repository is problematic because binary assets do not compress well and will bloat the size of our repository. **Proposal:** Binary test assets are packaged in to a nuget package per test assembly to be deployed at test run time. The binary assets to generate these nuget package are persisted in a single separate Git repository. Git repository structure: corefx-testdata\ corefx-testdata\System.IO.Compression.TestData.nuspec corefx-testdata\System.IO.Compression.TestData\ Nupkg structure: content\ Test asset nuget packages are depended on by the test assembly’s packages.config Currently, nuget is designed to deploy at package reference time (ie. when a package is referenced in Visual Studio) rather than runtime. In Dev14 this is changing. Until then, we’ll include a target in the test assembly’s csproj to copy the assets from the downloaded nuget package to the test’s working directory. In addition to System.IO.Compression, metadata reader uses binary test assets, but as they are only 100KB combined and are not a practical problem. System.IO.Packaging has 60GB of test assets. **Proposal:** System.IO.Packaging split their test assets in to tiers, with a single nuget package per tier. The inner loop tests should be no more than some number of megabytes. **_Native Dependencies**_ In the desktop .NET Framework, although we have a purely managed implementation of the deflate algorithm, customers have reported that the popular cross-platform open-source zlib library is faster and performs high quality compression. In response we include zlib, which we wrap and delegate to. We build it from source using the unmodified 1.2.3 version of the sources in the FX partition, and name the resulting output “clrcompression.dll.” On Windows, to fit with .NET Core’s app-local, side-by-side model, if we include zlib, we should deploy a copy of the native assembly alongside the managed System.IO.Compression.dll assembly included with an application. zlib distributes both sources and official binaries. While we could redistribute the official zlib binary, unfortunately they only release a 32-bit version and we are currently distributing a 64-bit version of coreclr. While zlib has a permissive license, it is not the MIT license, and including it in the corefx repository would mean the corefx repository no longer only contained code under a single license. **Proposal:** Continue building clrcompression.dll from the FX repository. Publish the 64-bit version of clrcompression.dll in a nuget package System.IO.Compression.clrcompression-x64.nupkg. Make System.IO.Compression.clrcompression-x64 a dependency of the System.IO.Compression package. **X-Plat todo:** Linux has a strong history of deploying applications and their dependencies through package managers. Packagers bundle the application binaries and metadata describing dependencies in to packages, and publish those on package manager repositories. We are planning to declare other framework and runtime dependencies through package managers where appropriate, and System.IO.Compression’s zlib dependency should be consistent. " 1305 area-System.Xml Inconsistent behavior of XmlAttributeCollection.InsertBefore and InsertAfter "I've noticed an inconsistent behavior of `XmlAttributeCollection.InsertAfter(XmlAttribute newNode, XmlAttribute refNode)` comparing to `InsertBefore` when the reference node is the last in the collection (the first for `InsertBefore`) and it's a duplicate of the new node. The duplicate node (attribute) needs to be removed before inserting. `InsertAfter` is unable to handle the case and throws `System.ArgumentOutOfRangeException` further in `List.Insert()` due to an incorrectly calculated insert position, while `InsertBefore` does the job. I'm not sure if this case has any practical use, but might be worth fixing. The following unit test reproduces the problem: ``` [Fact] // [ActiveIssue(""Throws argument out of range exception"")] public void InsertAfterRemovesDupRefAttr() { const string attributeName = ""existingAttr""; const string attributeUri = ""some:existingUri""; XmlDocument doc = CreateDocumentWithElement(); XmlElement element = doc.DocumentElement; element.Attributes.Append(doc.CreateAttribute(""attr1"", ""some:uri1"")); element.Attributes.Append(doc.CreateAttribute(""attr2"", ""some:uri2"")); XmlAttribute refAttr = element.Attributes.Append(doc.CreateAttribute(attributeName, attributeUri)); //dup XmlAttribute newAttr = doc.CreateAttribute(attributeName, attributeUri); XmlAttributeCollection target = element.Attributes; target.InsertAfter(newAttr, refAttr); // throws // Assert... } ``` A similar case (the reference attribute is the first in the collection) works for `InsertBefore`: ``` [Fact] public void InsertBeforeRemovesDupRefAttr() { const string attributeName = ""existingAttr""; const string attributeUri = ""some:existingUri""; XmlDocument doc = CreateDocumentWithElement(); XmlElement element = doc.DocumentElement; XmlAttribute refAttr = element.Attributes.Append(doc.CreateAttribute(attributeName, attributeUri)); //dup element.Attributes.Append(doc.CreateAttribute(""attr1"", ""some:uri1"")); element.Attributes.Append(doc.CreateAttribute(""attr2"", ""some:uri2"")); XmlAttribute newAttr = doc.CreateAttribute(attributeName, attributeUri); XmlAttributeCollection target = element.Attributes; target.InsertBefore(newAttr, refAttr); // Assert... } ``` " 1313 area-System.IO FileInfo: get_LastWritTime test fail ``` EXEC : warning : (runTest, d:\corefx\src\System.IO.FileSystem\tests\FileInfo\get_LastWriteTime.cs, 100) Eror_f984f! LastWriteTime is way off [D:\corefx\src\System.IO.FileSystem\tests\System.IO.FileSystem.Tests.csproj] FAiL! get_LastWriteTime.cs ,iCountErrors==1 FileInfo_get_LastWriteTime.runTest [FAIL] Assert.Equal() Failure Expected: 0 Actual: 1 Stack Trace: d:\corefx\src\System.IO.FileSystem\tests\FileInfo\get_LastWriteTime.cs(126,0): at FileInfo_get_LastWriteTime.runTest() ``` 1314 area-System.Collections Regression: Comparer_DefaultInvariant.ExecuteComparer_DefaultInvariant http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_release/lastFailedBuild/testReport/junit/(root)/Comparer_DefaultInvariant/ExecuteComparer_DefaultInvariant/ MESSAGE: Assert.True() Failure +++++++++++++++++++ STACK TRACE: at Comparer_DefaultInvariant.ExecuteComparer_DefaultInvariant() in d:\j\workspace\dotnet_corefx_windows_release\src\System.Collections.NonGeneric\tests\Comparer\Comparer_DefaultInvariant.cs:line 114 1316 area-Infrastructure Use roslyn compiler to build corefx We should be using the Roslyn compiler to build 1317 area-Serialization Remove LegacyNetCF serializer from XmlSerializer 1318 area-Infrastructure Single tests should be able to be run through the IDE Currently there seems to be some miscommunication between the corefx repo/project setup and the IDE, and the test discovery inside the IDE for xunit is broken. This makes it difficult to run single tests 1326 area-System.IO System.IO.Compression tests need to remove a workaround for not understood DST issue The System.IO.Comprssion tests verify the last modified metadata of zip file entries. There is currently a workaround where the check will succeed if the entry is 1 hour earlier than expected. This is to work around a not understood DST issue either in the test, or in the library. I need to understand if the issue is a test or product issue, and fix. 1330 area-System.Linq Parallel.Linq.Distinct not returning first element equal element in ordered queries. "Parallel.Linq.Distinct doen't return the first ""equal"" element in ordered queries. Take, for example, the following set: `{0, 1, 2, 3, 4, 5, 6, 7}` - and use a comparator that says everything `mod 4` is identical (so, `0` is the same as `4`, `1` and `5`, etc). I'm somewhat expecting to get `{0, 1, 2, 3}`, but I don't - any of the 'later' identical values can be returned. For those of you contending that ""They're identical, it doesn't matter"", which I somewhat thought too, I'm basing this on comments internal to the [Distinct operator](https://github.com/dotnet/corefx/blob/master/src/System.Linq.Parallel/src/System/Linq/Parallel/QueryOperators/Unary/DistinctQueryOperator.cs#L238-L251): ``` 238 // For each element, we track the smallest order key for that element that we saw so far 239 TKey oldEntry; 240 241 Wrapper wrappedElem = new Wrapper((TInputOutput)elem.First); 242 243 // If this is the first occurrence of this element, or the order key is lower than all keys we saw previously, 244 // update the order key for this element. 245 if (!_hashLookup.TryGetValue(wrappedElem, out oldEntry) || _keyComparer.Compare(orderKey, oldEntry) < 0) 246 { 247 // For each ""elem"" value, we store the smallest key, and the element value that had that key. 248 // Note that even though two element values are ""equal"" according to the EqualityComparer, 249 // we still cannot choose arbitrarily which of the two to yield. 250 _hashLookup[wrappedElem] = orderKey; 251 } ``` The [ordering page](https://msdn.microsoft.com/en-us/library/vstudio/dd460677%28v=vs.110%29.aspx) just says these operators return ""ordered"" results given an ordered source, which is technically still happening - so long as there was an element in that original position, that or an equal one is returned in that position. Vanilla LINQ actually says `Distinct` returns _unordered_ results, but I'm guessing they really meant ""we didn't use `OrderBy` to make things easier for us"", especially as all the examples (and the way the operator is implemented) show that it retains the ordering of the original source (and how the other set operators behave - `Except` doesn't mention ordering, `Intersect` explicitly mentions the original order is retained, and `Union` mostly implies the original order is retained). So... which is correct? Should the actual-first equal element be returned (as I'm expecting), or that's a stronger guarantee than we need (and any equal item can be returned in its place)? Example reproduction code (may fail to fail): ``` using System; using System.Collections.Generic; using System.Linq; using Xunit; namespace Test { public class Example { [Fact] public static void DistinctExample() { ParallelQuery original = ParallelEnumerable.Range(0, 24).AsOrdered(); int seen = 0; Assert.All(original.Distinct(new ModularCongruenceComparer(4)), i => Assert.Equal(seen++, i)); Assert.Equal(4, seen); } private class ModularCongruenceComparer : IEqualityComparer { private int _mod; public ModularCongruenceComparer(int mod) { _mod = Math.Max(1, mod); } private int leastPositiveResidue(int x) { return ((x % _mod) + _mod) % _mod; } public bool Equals(int x, int y) { return leastPositiveResidue(x) == leastPositiveResidue(y); } public int GetHashCode(int x) { return leastPositiveResidue(x).GetHashCode(); } public int GetHashCode(object obj) { return GetHashCode((int)obj); } } } } ``` " 1332 area-System.Linq Parallel.Linq.Concat not consistent with ordered input "`Parallel.Linq.Concat` isn't behaving consistently when given ordered input sources. As `Concat` is a binary operator, two sources may be supplied. The inconsistency surfaces depending on which, and how, operand is ordered. If both **or** either (only the left or the right) is ordered via `AsOrdered()`, the entire output is ordered. If the left operator is ordered via any of the `OrderBy(...)` operators (`OrderBy`, `OrderByDescending`, `ThenBy`, `ThenByDescending`), the entire output is ordered. If the _right_ operator is ordered via any of the `OrderBy(...)` operators, only the _left_ output is ordered (as by `AsOrdered()`), and the right is returned in a non-deterministic order. It only seems to do this with right sources constructed via `Enumerable.Range(...).AsParallel()` and `Partitioner.Create(...).AsParallel()`. Using a `List` is safe (eg, `Enumerable.Range(...).ToList().AsParallel()`), however the output of LINQ is not: `list.Select(x => x).AsParallel()` can return unordered output. How the left operand is constructed appears to have no effect. This is also the behavior of the current consumer release. Example program: ``` static void Main(string[] args) { int leftSize = 8; int rightSize = 8; ParallelQuery query = ParallelEnumerable.Range(0, leftSize) .Concat(Enumerable.Range(0, rightSize).AsParallel().OrderBy(x => x)); foreach (int i in query) { Console.Write(i + ""|""); } Console.ReadLine(); } ``` (This is a bit of a Heisenbug - expect it to generate completely ordered output on occasion. It's possible that it does this for all input types, and I just haven't seen it yet, although I consider this unlikely) The [ordering page](https://msdn.microsoft.com/en-us/library/vstudio/dd460677%28v=vs.110%29.aspx) states that ordered input to `Concat` results in ordered output, and unordered input results in unordered output, but it isn't clear what happens when you have one of each. At minimum, the output actually observed is disturbing on first encounter. `Zip` appears to operate similar to how I anticipate `Concat` operating - if either input is ordered, the entire output is ordered, regardless of source. " 1334 area-System.IO Investigate async usage for System.IO.Compression "@stephentoub pointed out two suspect uses of Async in System.IO.Compression in https://github.com/dotnet/corefx/pull/1309 - In System\IO\Compression\DeflateStream.cs: ``` 347 + // If there is no data on the output buffer and we are not at 348 + // the end of the stream, we need to get more data from the base stream 349 + readTask = _stream.ReadAsync(buffer, 0, buffer.Length, cancellationToken); 350 + if (readTask == null) 351 + { 352 + throw new InvalidOperationException(SR.NotReadableStream); 353 + } 354 + 355 + var tcs = new TaskCompletionSource(); 356 + 357 + // ContinueWith will never throw here otherwise we'll be in inconsistent state 358 + readTask.ContinueWith( ``` ""It's not clear to me why ContinueWith is being used in this fashion rather than using async/await. We should open a work item to revisit this async implementation."" In System\IO\Compression\DeflateStream.cs: ``` 625 + Interlocked.Increment(ref asyncOperations); 626 + 627 + try 628 + { 629 + return base.WriteAsync(array, offset, count, cancellationToken).ContinueWith( 630 + (t) => Interlocked.Decrement(ref asyncOperations), ``` ""This looks like a real bug. Any exceptions or cancellation information from the Task returned from the base WriteAsync is getting eaten."" " 1336 area-System.Linq Collections should have extensions to compute order statistic `System.Linq` has extensions implemented for computing the minimum and maximum. Example: ``` C# var a = new int[] {100, 300, 200, 400}; WriteLine(a.Max()); // prints 400 WriteLine(a.Min()); // prints 100 ``` While these are useful values when dealing with collections, they are merely special cases of nth order statistics. More often than not, people are also interested in a specific cases. For example, when querying the collection in the sample code above, the question may be posed, what is the 3rd smallest item in the array? The naive implementation would be to sort the array and print the 3rd item in it. However, there are selection algorithms that can do this more efficiently, without the need to actually invoke a sort method on the array. A method should be implemented in line with the `Min` and `Max` extension methods to accomplish this. The API should be a simple as: ``` C# WriteLine(a.NthSmallestElement(3)) // should print 300 WriteLine(a.NthSmallestElement(1)) // should print 100 WriteLine(a.NthSmallestElement(a.Length)) // should print 400 ``` The method name may differ based on further discussion. I chose the name based on C++ references that I had seen earlier. 1343 area-Serialization Add serialization support for XmlDocument and XmlElement in XmlSerializer Add back the code that was removed when moving serialization to .NET Core. Issue #1256. 1346 area-System.Runtime Support for ICustomMarshaler Split from #1333 Recorded uses in (3rd party) libraries: - [LibGit2Sharp](https://github.com/libgit2/libgit2sharp/search?utf8=%E2%9C%93&q=ICustomMarshaler&type=Code) - Mono.Posix see #94 1347 area-Serialization Support for binary serialization Split from #1333 Turns out binary serialization is not really needed in my case as it's used only on exceptions, but it would still be nice to know whether it's coming back or whether it is depreciated. 1349 area-System.Reflection Dev/metadata Merge changes from master. 1351 area-Serialization Add serialization support for XmlDocument and XmlElement in XmlSerializer Add back the code that was removed when moving serialization to .NET Core. Issue #1256. 1355 area-System.IO Failure in System.IO.FileSystem.DriveInfo ``` Discovering: System.IO.FileSystem.DriveInfo.Tests Discovered: System.IO.FileSystem.DriveInfo.Tests Starting: System.IO.FileSystem.DriveInfo.Tests System.IO.FileSystem.DriveInfoTests.DriveInfoWindowsTests.TestVolumeLabel [FAIL] Assert.Throws() Failure Expected: typeof(System.UnauthorizedAccessException) Actual: typeof(System.IO.IOException): The request is not supported Stack Trace: e:\oss\corefx\src\System.IO.FileSystem.DriveInfo\src\System\IO\DriveInfo.Windows.cs(235,0): at System.IO.DriveInfo.set_VolumeLabel(String value) e:\oss\corefx\src\System.IO.FileSystem.DriveInfo\tests\DriveInfo.Windows.Tests.cs(180,0): at System.IO.FileSystem.DriveInfoTests.DriveInfoWindowsTests.<>c__DisplayClass20.b__18() Finished: System.IO.FileSystem.DriveInfo.Tests ``` I think this is consistent failure on my box (at least repeated few times) This happens for my DVD drive when there is no drive inside. @Priya91 seemed to also have checked it on her computer but she was unable to repro. When I use DriveInfo class on my DVD drive it is able to get the DriveType but trying to access VolumeLabel or DriveFormat (also for read) causes the same failure. 1360 area-System.Collections System.Collections.Specialized.Tests.GetItemObjTests.Test01 [FAIL] ``` System.Collections.Specialized.Tests.GetItemObjTests.Test01 [FAIL] Error, returned wrong item for uppercase key Stack Trace: d:\j\workspace\dotnet_corefx_windows_release_prtest\src\System.Collections.Specialized\tests\HybridDictionary\GetItemObjTests.cs(328,0): at System.Collections.Specialized.Tests.GetItemObjTests.Test01() ``` I coudln't repro locally, probably some threading issue 1370 area-System.Diagnostics Provide an API for traversing/reasoning about async call stacks Any tool that shows exceptions today also shows the callstack. With the introduction of the async keyword (yield was the same I guess), it's becoming increasingly harder to show reasonable call stacks because of the async state machine. It would be nice if there was an API somewhere that could be used to clean up the async state machine and pretty print a call stack with readable method names (tools like reflector do this). Here's an example of a typical error page in an MVC 6 application: ![image](https://cloud.githubusercontent.com/assets/95136/7085404/4e19da18-df2b-11e4-8fa7-28a0799854f0.png) 1373 area-System.Linq enumerable.AsParallel().LongCount() can only count int.MaxValue elements "... as well as `enumerable.AsParallel().LongCount(x => true)` (I thought this override had a successful run, but it isn't now) The overflow is encountered at [PartitionedDataSource.cs, line 967](https://github.com/dotnet/corefx/blob/master/src/System.Linq.Parallel/src/System/Linq/Parallel/Partitioning/PartitionedDataSource.cs#L697). The problem is that these partitioned data sources all presume to only operate over `int.MaxValue` elements, and the rest of the file (and related infrastructure) depend on that. Some sort of re-architecting (rather than simply switching the key from `int` to `long`) is probably required. Repro test: ``` [Fact] // Extremely long-running public static void LongCount_LargeCount() { Assert.Equal(int.MaxValue + 1L, new OverlyLargeEnumerable().AsParallel().LongCount()); } private class OverlyLargeEnumerable : IEnumerable { public IEnumerator GetEnumerator() { return new OverlyLargeEnumerator(); } IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); } private class OverlyLargeEnumerator : IEnumerator { private long _current = -1; public long Current { get { return _current; } } object IEnumerator.Current { get { return Current; } } public void Dispose() { _current = int.MaxValue; } public bool MoveNext() { // Adds one _after_ compare, so final value is int.MaxValue + 1 return _current++ < int.MaxValue; } public void Reset() { _current = -1; } } } ``` Vanilla LINQ is fine. This behavior is present on the current consumer release. (""You had one job! ONE! JOB!"") (side note - I'm a little surprised by how much slower this runs in parallel compared to doing it sequentially - by about 6 times, or ~120 seconds vs 20 seconds) " 1378 area-System.Collections Add IEnumerable> variants to Dictionary's .ctors We should consider adding constructors to `Dictionary` which allow building a Dictionary from an IEnumerable of KeyValueParis. ## Rationale and Usage Given an `IEnumerable>` there's no simple way to construct a dictionary, around it. Instead you have to do something like: ``` csharp var dict = new Dictionary(); foreach (KeyValuePair kvp in enum) { dict.Add(kvp.Key, kvp.Value); } ``` or ``` csharp var dict = enumerable.ToDictionary(kvp => kvp.Key, kvp => kvp.Value); ``` `ConcurrentDictionary` already provides such .ctors. This is especially useful in the case where you want to make a copy of an `IReadOnlyDictionary` ## Proposed API ``` csharp public class Dictionary : ICollection, ICollection>, IDictionary, IDictionary, IEnumerable, IEnumerable>, IReadOnlyCollection>, IReadOnlyDictionary { // New .ctors public Dictionary(IEnumerable> enumerable); public Dictionary(IEnumerable> enumerable, IEqualityComparer comparer); // Existing .ctors public Dictionary(); public Dictionary(IDictionary dictionary); public Dictionary(IDictionary dictionary, IEqualityComparer comparer); public Dictionary(IEqualityComparer comparer); public Dictionary(int capacity); public Dictionary(int capacity, IEqualityComparer comparer); // Remaining members removed for brevity. } ``` See some additional discussion on this issue is at dotnet/coreclr#680 1389 area-System.Reflection Fixes a couple of bugs in debug handle enumerators 1414 area-System.Collections ImmutableList (and Builder) IList.Contains, IndexOf, Remove methods throw instead of return -1 given unexpected argument types A bit longer story http://blog.rthand.com/post/2015/04/10/A-hidden-bug-in-Windows-Stores-ComboBox.aspx Basically a Windows Store (8.1 and 10 at least) ComboBox first tries ComboBoxItem against its ItemsSource IList.IndexOf and if that gracefully fails (returns -1) then it tries with the actual type. This works with normal List because this guy returns -1 when an incompatible type is used, where ImmutableList throws an exception (doesn't do any check actually) and ComboBox doesn't understand that very well. To make it worse ComboBox doesn't perform this stunt immediately but only after a while (when one scrolles some of its items). Now, I'd say that this is ComboBox' fault but who knows. 1415 area-System.Linq Propose: new method Enumerable.IndexOf Hi, I would like to propose a new member for Enumerable class: ``` int IndexOf(IEnumerable source, T item, IEqualityComperer comparer) ``` It's useful to find the index of an item inside the source. Only `IList` has a similar method but it forces me to completely materialize all the source. The implementation can enumerate list or, if the original source type is `IList`, call the `IList.IndexOf` method. If the source type is an `Array` we can call `Array.IndexOf`. What do you think? 1440 area-Serialization Improve code coverage for data contract serializers Improve code coverage for DataContractSerializer and DataContractJsonSerializer 1441 area-Serialization Expose more details in serialization test failures Include expected and actual output when serialization tests fail and clean up tests to get rid of Console.WriteLine. 1443 area-System.Reflection Merge master 1450 area-System.Runtime Make TimeZoneInfo.AdjustmentRule.BaseUtcOffsetDelta public I would like to propose that `TimeZoneInfo.AdjustmentRule.BaseUtcOffsetDelta` be marked `public` instead of `internal`. This can be seen [here](https://github.com/dotnet/coreclr/blob/d46b09d4dfc8e25d7e61a9994f89720cf01b5086/src/mscorlib/src/System/TimeZoneInfo.cs#L3754). There are several reasons: - The `AdjustmentRule` class is loaded from registry information, loosely matching the `_REG_TZI_FORMAT` structure. (See [this article](http://blogs.msdn.com/b/bclteam/archive/2007/06/07/exploring-windows-time-zones-with-system-timezoneinfo-josh-free.aspx)). - The whole reason this property was added ([KB3012229](https://support.microsoft.com/en-us/kb/3012229)) is such that the data from the registry can be better reflected in this class. Specifically, the `_REG_TZI_FORMAT.Bias` value can change from year to year, so `AdjustmentRule.BasUtcOffsetDelta` was created to track the delta between the year-to-year bias and the default bias for the time zone. - There's no good reason (IMHO) to keep this internal. Doing so means that the full functionality of the `AdjustmentRule` class can really only be used when populated by the framework itself. It also makes understanding the way that the framework populates the class difficult, as all fields are exposed through public properties _except_ this one. - It creates problems for those trying to use it in other scenarios. See Example 2 in Jon Skeet's post [Backward Compatibility Pain](http://codeblog.jonskeet.uk/2015/03/02/backward-compatibility-pain/). Additionally, there needs to be some public way to create an `AdjustmentRule` that has a value for this property. Either the existing `CreateAdjustmentRule` factory method should be made public, or a parameter could be added to the existing constructor, or another constructor could be added. Thanks for your consideration. 1455 area-System.Reflection Merge branch 'master' of https://github.com/dotnet/corefx into dev/metadata 1464 area-System.Reflection Set genericParameterCount in MethodSignature This addresses issue #1463 1472 area-System.Reflection Merge branch 'master' of https://github.com/dotnet/corefx into dev/metadata 1474 area-Meta CoreFX Primer "I'm writing a [.NET Primer](https://github.com/dotnet/coreclr/pull/725) in the CoreCLR repo that has a strong bias to the runtime features. I'd like to start one that has a strong bias to the framework libraries. It makes sense for it to live in this repo. Naturally, there would be cross-linking between the two. - Good idea? - What should this primer be called? I don't want to call one "".NET Core Runtime Primer"" and then other "".NET Core Framework Primer"". The primer I'm working on right now is the primer for the fundamentals of .NET generally, in an ECMA 335 sense. It's intended to be equally meaningful for Mono. The CoreFX Primer, on the other hand, would be specific to this implementation, so we could reasonable call it the "".NET Core Framework Primer"". /cc @Petermarcu FYI: This topic would move the corefx primer: https://github.com/dotnet/coreclr/blob/dotnet-primer/Documentation/framework-libraries.md. " 1476 area-System.Text Regex constructor throws NullReferenceException and OutOfMemoryException "Following line of code throws NullReferenceException: Regex regex = new Regex(""(?(?i))""); And following line of code throws OutOfMemoryException: Regex regex = new Regex("" (?(?i))""); Both patterns are if constructs where expression ""?i"" contains quantifier following nothing. There seems to a problem with ""i"" letter or any other letter that is an inline character for a particular option (i, m, n, s, x). If you change ""i"" with ""a"" (not an inline character), ArgumentException is thrown as expected. Both patterns are invalid so according to the .NET documentation, both lines should throw ArgumentException. This behaviour is especially unpleasant in case of ""regex tester"" application (such as http://regexator.net) which tries to continually evaluate user input as regex pattern. This behaviour has been tested in .NET 4.0, 4.5 and 4.6, all with the same results. " 1477 area-Infrastructure Test infrastructure requirements and scenarios "Current updated proposal from @Chrisboh : **Requirements we need to satisfy** • Run a set of tests that all developers should run before checking in • Specify a set of tests to run from the command line • Be able to debug all test cases • Run with code coverage turned on • Only build what is required ## Proposal The key to this proposal starts at build time. We need to only build what we are going to run. This is important both for overall runtime as well running tests using .net native. In order to satisfy this we need to move our ability to filter up higher in the process. ## Building only What is Required The following include is where we find all of our test projects and add them to the list of items we would like to build. **File:** CoreFX\src\dirs.proj (line 8) `` We need to add a condition to this to only include items that match a supplied filter. This filter would contain the following items. **Filter List:** • InnerLoop • Functional • Partner • Customer #### InnerLoop This will contain everything we consider an Unit test today and until we complete the work to run our tests based on selectivity should be the minimum required set of tests that get run before accepting any PR. #### Functional (OuterLoop Ring 1) This category is made up of test cases that will test larger sections of the code and / or take longer to run. The tests in this category should also give us a quick high level check that there are no major issues with regards to basic performance, stress, or security. #### Partner (OuterLoop Ring 2) The goal of this category is to determine if we are ready for team Dogfooding. All performance and stress tests should be in this category as well as any compliance / security tests. #### Customer (OuterLoop Ring 3) The goal of this category is to determine if we are ready to release this for Partner Dogfooding. This will contain all scenario test cases and will have a manual component added as well. ## Defining Test Project In order for our filtering to work correctly we need to ensure that test project is setup correctly. The simple way to do this is to create a test project for each category in the Filter list. This would give us at most 4 test projects but it will be more typical to have around 2 projects as I could see unifying scenario, compliance, and security tests in a single location. In order to do this we will need to filter by test project name. We will follow the following naming conventions. `*..tests.dll` Example: `System.ObjectModel.InnerLoop.tests.dll` ## Xunit Attributes Now it is time to address what we are going to be doing about the Xunit attributes. First we will remove all attributes that can be replaced with the project filters. This will include things like InnerLoop, OuterLoop, Performance, and Stress. These attributes will not be used when deciding what binaries we should build and run. What this doesn't mean is that we will be removing all attributes entirely. We still need to keep and leverage certain attributes in order to give us the flexibility we need to further fine tune our runs. Here are some of the key attributes we will need to leave in place. #### Active Issue Attribute This attribute will be used to filter out failing test cases when running locally. When running our daily builds we should include these test cases and label them as failing with a known bug. #### Platform Specific Attribute This attribute will be used to filter out test cases that do not apply to the platform you are currently running on. There was some debate on whether we should promote this to the filter list or not. At this time we don’t feel there are enough test cases to warrant promoting this to the filter list. In the future if this becomes an issue we can revisit this. ## Command Line We will need to execute everything from the command line as well as be able to select everything we need. Doing so will be done via two properties. The first will be defining the test categories that you would like to include. This will be done with the following property. `/p:TestTypes=""InnerLoop;Functional""` The second property will be used to supply extra command line parameters to Xunit. This will simply be a pass-through from the msbuild command line onto Xunit and we will use the following property for that. `/p:AdditionalXunitArgs=""-notrait ActiveIssue""` Additionally if you do not specify the TestTypes property it should run all Developer test cases automatically. Also not specifying the AdditionalXunitArgs property should be default exclude any test with an Active Issue and regardless if the property is specified we should include the correct OS parameter. ## Debugging This will be the same as what we have today. In the future we should invest me making the Xunit.runner.VisualStudio work and this would simplify everything. " 1490 area-System.Linq Parallel.Linq.GroupBy not using first 'equal' key. Similar to `Distinct` ( #1330 ), `GroupBy` isn't choosing the 'first' or earliest identical element for a key. That is, given an ordered range `{0, ... 23}`and running `GroupBy(x => x, compareMod(4))`, I'm expecting the first (lowest, here) element that occurs in the source to be chosen as the key - `{0, 1, 2, 3}`. Instead, any element may be chosen as the key (although it looks like `0` is consistently being chosen for the first key... hmmm...) Reproduction code (may not always fail): ``` using System; using System.Collections.Generic; using System.Linq; using Xunit; namespace Test { public class Example { [Fact] public static void GroupByExample() { ParallelQuery original = ParallelEnumerable.Range(0, 24).AsOrdered(); int seen = 0; Assert.All(original.GroupBy(x => x, new ModularCongruenceComparer(4)), i => Assert.Equal(seen++, i.Key)); Assert.Equal(4, seen); } private class ModularCongruenceComparer : IEqualityComparer { private int _mod; public ModularCongruenceComparer(int mod) { _mod = Math.Max(1, mod); } private int leastPositiveResidue(int x) { return ((x % _mod) + _mod) % _mod; } public bool Equals(int x, int y) { return leastPositiveResidue(x) == leastPositiveResidue(y); } public int GetHashCode(int x) { return leastPositiveResidue(x).GetHashCode(); } public int GetHashCode(object obj) { return GetHashCode((int)obj); } } } } ``` 1491 area-System.Xml Some tests are failing when decimal separator is not a dot Some XML tests are failing if decimal separator is not a dot. Example repro: - Start->Run (WinKey + R) -> `intl.cpl` - Change format (no need to change locale) to `Ukrainian (Ukraine)` - `build /p:WithCategories=OuterLoop` Example error: ``` TestCase:WriteFullEndElement - ERROR: RECA Expected: -4582,24 Actual : -4582.24 elem.WriteValue(floatToUri) ``` 1498 area-System.Linq Unit tests for Enumerable.Where Following MarcinJuraszek's examples for Enumerable.Select, I added basically identical tests for Enumerable.Where. 1502 area-System.Collections [Proposal] Use B+-trees instead of AVL trees for Immutable Collections Currently the `Node` class used for `ImmutableList` includes 40 bytes of overhead per list element on X64, and 24 bytes of overhead on X86. This means the memory cost of using this data structure for an `ImmutableList` is 1000% on X64, and 600% on X86 (these figures represent the amount of memory used for storing non-data elements associated with the structure). I propose migrating to a B+-tree instead, using a fixed value for _b_ and declaring data values in the index and data pages directly (to avoid unnecessary memory overhead for arrays). The choice of _b_ primarily affects the following things: 1. The height of the tree. Shorter tree heights improve performance for random access lookups by improving memory locality for a majority of traversal operations. 2. The size of the pages. Mutations of immutable data structures require reconstructing pages leading back to the root. If the pages are large, mutations could become expensive. In addition to the above, B+-trees provide a substantial memory savings by removing the child pointers (the Left and Right pointers for AVL nodes) from the leaves. My initial calculations (which could be completely wrong) indicate that the average memory allocation required for a single mutation operation on a list of 100000 elements using the AVL structure is 775 bytes for X64 and 493 bytes for X86. The optimal value of _b_ for X64 in terms of mutation allocation overhead is _b_=8, which costs 476 bytes per operation on a list of the same size (39% reduction). For X86 this same value of _b_ costs 388 bytes per mutation (21% reduction, and _b_=6 would be optimal with 382 bytes). The overall storage overhead for _b_=8 drops a staggering amount down to 169% for X64 and 119% for X86 (counting the average page at 75% capacity, based on a merge at 50% and split at 100%). 1504 area-System.Collections Some Collections.NonGeneric tests have race conditions and ocassionally failing ( http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/963/console ) ``` Xunit.Sdk.EqualException: Assert.Equal() Failure Γåô (pos 0) Expected: en-US Actual: cs-CZ Γåæ (pos 0) at Xunit.Assert.Equal(String expected, String actual, Boolean ignoreCase, Boolean ignoreLineEndingDifferences, Boolean ignoreWhiteSpaceDifferences) at Xunit.Assert.Equal(String expected, String actual) at Comparer_DefaultInvariant.runTest() in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Collections.NonGeneric\tests\Comparer\Comparer_DefaultInvariant.cs:line 52 Fail! iCountErrors==1 Comparer_DefaultInvariant.ExecuteComparer_DefaultInvariant [FAIL] Assert.True() Failure ``` The tests are setting: ``` csharp CultureInfo.DefaultThreadCurrentCulture = culture; CultureInfo.DefaultThreadCurrentUICulture = culture; ``` Which may fail when run on different threads. 1509 area-System.Numerics Poor hash function in .Net source The `CombineHashCodes` in `System.Numerics.HashCodeHelper` has a very high collision rate for uniformly distributed data! The method probably first appeared in `System.Web.Util.HashCodeCombiner` class and then used in `System.Array`, `System.Tuple` and dotnet/corefx. Here's an example: Assuming the `CombineHashCodes` method's implementation is: ``` C# internal static int CombineHashCodes(int h1, int h2) { return (((h1 << 5) + h1) ^ h2); } ``` 100K raw data results in more than 6K duplicate hash values, that is 6% of a uniformly distributed set: ``` C# var data = Enumerable.Range(0, 100000) .Select(t => new { Val1 = t, Val2 = 100000 + t }) .GroupBy(t => CombineHashCodes(t.Val1.GetHashCode(), t.Val2.GetHashCode())) .Where(t => t.Count() > 1) .Select(t => new { Hash = t.Key, Items = t.ToList() }) .ToList(); ``` I believe there are much better hash functions to use like the one Josh Bloch suggests in Effective Java, which results in no hash collision even for 1M records of data in example above: ``` C# internal static int CombineHashCodes(int h1, int h2) { return (17 * 31 + h1) * 31 + h2; } ``` 1511 area-Infrastructure Remove properties not relevant to code formatting from corefx.vssettings "Looks like we have a few properties that are unrelated to formatting in corefx.vssettings such as: ``` 0 2 ``` We should remove them so that only C# formatting settings are in that file. " 1512 area-System.Collections System.Collections.Immutable and APTCA/SecurityTransparentAttribute If possible please add one of attributes forcing assembly to be Security Transparent (AllowPartiallyTrustedCallersAttribute or SecurityTransparentAttribute). This will help using this library in some quite uncommon partial trust scenarios. 1513 area-System.Runtime Add String.Split overloads that take a single char and string separator It's extremely common to split a string based on a single `char` or `string` separator, yet `String.Split` only offers overloads that accept an array of separators. The `params char[] separator` overload is particularly insidious as the `params` usage often results in a large number of unnecessary `char[]` heap allocations, unbeknownst to the developer. ## Rationale and Usage [Stack Exchange ran into this](http://blog.marcgravell.com/2013/11/allocaction-allocation-allocation.html) (significant number of separator arrays in memory) and submitted a [pull request](https://aspnetwebstack.codeplex.com/SourceControl/network/forks/marcgravell/outputcachecleanup/contribution/7201) to ASP.NET MVC to cache `String.Split` separator arrays inside the ASP.NET MVC codebase. They ended up with the following: ``` c# namespace System.Web.Mvc { internal static class StringSplits { // note array contents not technically read-only; just... don't edit them! internal static readonly char[] Period = new[] { '.' }, Comma = new[] { ',' }; } } ``` With uses of `string.Split('.')` and `string.Split(',')` replaced with `string.Split(StringSplits.Period)` and `string.Split(StringSplits.Comma)` to avoid the `char[]` allocations. It'd be awesome if `String.Split` offered overloads that accepted a single separator to avoid this. ## Proposed API ``` c# public partial class String { // Proposed methods public string[] Split(char separator, StringSplitOptions options = StringSplitOptions.None); public string[] Split(char separator, int count, StringSplitOptions options = StringSplitOptions.None); public string[] Split(string separator, StringSplitOptions options = StringSplitOptions.None); public string[] Split(string separator, int count, StringSplitOptions options = StringSplitOptions.None); // Existing methods public string[] Split(params char[] separator); public string[] Split(char[] separator, int count); public string[] Split(char[] separator, StringSplitOptions options); public string[] Split(char[] separator, int count, StringSplitOptions options); public string[] Split(string[] separator, StringSplitOptions options); public string[] Split(string[] separator, int count, StringSplitOptions options); } ``` ## Notes We had wanted to add `public string[] Split(string separator)`, but this breaks source compatibility with uses of `Split(null)`, which is documented to split based on white space, because it makes the call ambiguous between `Split(char[])` and `Split(string)`. I won't go into it here as it's really a separate feature request, but it would also be worth considering new `Split` methods that return a `struct` collection of `StringSpan`/`StringSegment` structs (in lieu of internal runtime span/substring magic) to avoid the resulting `string[]` allocation (and substring allocations, unless needed). I'd be happy to contribute an implementation and tests. 1514 area-System.Runtime Add String.Format overloads to avoid unnecessary allocations "It's pretty common to pass value types to `String.Format`, unfortunately this results in unnecessary boxing allocations. ## Rationale Consider the following: ``` c# long id = ...; int index = ...; int length = ...; // Traditional call to String.Format string foo = string.Format(""{0}: Index = {1}, Length = {2}"", id, index, length); // Use of the string interpolation language feature string bar = $""{id}: Index = {index}, Length = {length}""; ``` Both the traditional call to `String.Format` and the use of the string interpolation language feature (which is just syntactic sugar for `String.Format`) requires 3 boxing allocations. ## Proposed API ``` c# namespace System { public sealed class String : ... { // Proposed methods public static string Format(string format, T0 arg0); public static string Format(string format, T0 arg0, T1 arg1); public static string Format(string format, T0 arg0, T1 arg1, T2 arg2); public static string Format(IFormatProvider provider, string format, T0 arg0); public static string Format(IFormatProvider provider, string format, T0 arg0, T1 arg1); public static string Format(IFormatProvider provider, string format, T0 arg0, T1 arg1, T2 arg2); // Existing methods public static string Format(string format, object arg0); public static string Format(string format, object arg0, object arg1); public static string Format(string format, object arg0, object arg1, object arg2); public static string Format(string format, params object[] args); public static string Format(IFormatProvider provider, string format, object arg0); public static string Format(IFormatProvider provider, string format, object arg0, object arg1); public static string Format(IFormatProvider provider, string format, object arg0, object arg1, object arg2); public static string Format(IFormatProvider provider, string format, params object[] args); ... } } namespace System.Text { public sealed class StringBuilder { // Proposed methods public StringBuilder AppendFormat(string format, T0 arg0); public StringBuilder AppendFormat(string format, T0 arg0, T1 arg1); public StringBuilder AppendFormat(string format, T0 arg0, T1 arg1, T2 arg2); public StringBuilder AppendFormat(IFormatProvider provider, string format, T0 arg0); public StringBuilder AppendFormat(IFormatProvider provider, string format, T0 arg0, T1 arg1); public StringBuilder AppendFormat(IFormatProvider provider, string format, T0 arg0, T1 arg1, T2 arg2); // Existing methods public StringBuilder AppendFormat(string format, object arg0); public StringBuilder AppendFormat(string format, object arg0, object arg1); public StringBuilder AppendFormat(string format, object arg0, object arg1, object arg2); public StringBuilder AppendFormat(string format, params object[] args); public StringBuilder AppendFormat(IFormatProvider provider, string format, object arg0); public StringBuilder AppendFormat(IFormatProvider provider, string format, object arg0, object arg1); public StringBuilder AppendFormat(IFormatProvider provider, string format, object arg0, object arg1, object arg2); public StringBuilder AppendFormat(IFormatProvider provider, string format, params object[] args); ... } } namespace System.Runtime.CompilerServices { public static class FormattableStringFactory { // Proposed methods public static FormattableString Create(string format, T0 argument0); public static FormattableString Create(string format, T0 argument0, T1 argument1); public static FormattableString Create(string format, T0 argument0, T1 argument1, T2 argument2); // Existing method public static FormattableString Create(string format, params object[] arguments); } } ``` I'd be happy to contribute an implementation and tests. " 1533 area-System.IO FileStream constructor should allow bufferSize = 0 to indicate non-buffered stream FileStream.Init will throw ArgumentOutOfRangeException if (bufferSize <-0), IMHO it should allow bufferSize = 0 value, indicating this FileStream object nerver uses buffer on this level, provides more flexibility and control for app developer, current workaround is set bufferSize to 1, which is ugly. 1534 area-System.Collections Should FEATURE_RANDOMIZED_STRING_HASHING be set in corefx For collection types in coreclr, we use randomized string hashing above a threshold. This is guarded with the feature define FEATURE_RANDOMIZED_STRING_HASHING. If we use the same behavior in corefx for types including HashSet and HashTable, we need to port its supporting code. 1538 area-System.Diagnostics Fix the System.Diagnostics.Process tests on OS X Some System.Diagnostics.Process tests are failing on OS X; root cause the failures and fix whatever is breaking 1543 area-System.Reflection Fix decoding of multi-mod types in signatures Address issue #1535 by updating the metadata reader's signature decoder to correctly decode types with multiple modifiers. 1546 area-Serialization Fix security exception with private serialization event methods in DCJS "Fix security exception in the deserialization scenario of type with serialization event method marked as private, such as: ``` csharp [DataContract] public class Data { [OnDeserialized] private void OnDeserialized(StreamingContext context) { } [DataMember] public int Property1 { get; set; } } … string json = ""{\""Property1\"":1}""; DataContractJsonSerializer s = new DataContractJsonSerializer(typeof(Data)); Data obj = s.ReadObject(new MemoryStream(Encoding.UTF8.GetBytes(json))) as Data; //Crashes ``` Actual behavior: SecurityException: The data contract type 'DataContractInternalRepro.Data' cannot be deserialized because the OnDeserialized method 'OnDeserialized' is not public. Making the method public will fix this error. Expected behavior: no exception This scenario is working in DCS. It looks like the exception in DCJS is thrown at the wrong place. " 1552 area-System.Reflection Merge 1561 area-System.Linq LINQ's Buffer ctor requires Count/CopyTo results be consistent "LINQ's internal Buffer, which is used in operations like ToArray, OrderBy, Reverse, etc., assumes that if the input `IEnumerable` implements `ICollection`, then CopyTo will be guaranteed to copy the number of items returned previously by Count: https://github.com/dotnet/corefx/blob/master/src/System.Linq/src/System/Linq/Enumerable.cs#L3087-L3093 ``` C# int count = 0; ICollection collection = source as ICollection; if (collection != null) { count = collection.Count; if (count > 0) { items = new TElement[count]; collection.CopyTo(items, 0); } } else { foreach (TElement item in source) { if (items == null) { items = new TElement[4]; } else if (items.Length == count) { TElement[] newItems = ArrayT.Resize(items, checked(count * 2), count); items = newItems; } items[count] = item; count++; } } ``` This is problematic for any collection that fails this guarantee, e.g. by design the concurrent collections (`ConcurrentQueue`, `ConcurrentDictionary`, etc.) may have their number of elements change between the call to Count and CopyTo if elements were added or removed concurrently. We could address this in the implementation of Buffer by avoiding the use of CopyTo, e.g. ``` C# int count = 4; ICollection collection = source as ICollection; if (collection != null) { int tmpCount = collection.Count; if (tmpCount > 0) count = tmpCount; } foreach (TElement item in source) { if (items == null) { items = new TElement[count]; } else if (items.Length == count) { TElement[] newItems = ArrayT.Resize(items, checked(count * 2), count); items = newItems; } items[count] = item; count++; } ``` This turns Count into an estimate rather than a guarantee, avoiding the problem. However, it has the downside of having a (potentially) less efficient copy mechanism for other `ICollection` implementations, including an `IEnumerator` allocation used in the foreach. That said, it's not clear to me how often we hit the ""fast"" CopyTo path anyway; many collection types have their own ToArray method, so if the LINQ method is being used, it's likely because there are other LINQ operators in the middle, e.g. `collection.Where(...).Select(...).ToArray()`, in which case the optimization will not apply. A few possible options: 1. We say that concurrent collections may not be used concurrently with LINQ operations. Developers may of course already be doing this, and since you can pass concurrent collections around as `IEnumerable`, the developer may not even know they're using concurrent collections but may silently either get corrupted data (e.g. the count drops after allocating the array, leaving default elements at the end of the array) or exceptions (e.g. CopyTo tries to copy more elements than the array can hold). 2. We make the above change, such that we no longer require the Count == CopyTo size guarantee, potentially special casing some known types like `List`, `Collection`, etc. and some known other situations, e.g. IsReadOnly returning true. 3. We special-case the known-problematic cases in the framework, namely the Concurrent\* collections, and do the second option above just for those. We could add a marker interface or something to more easily identify them, which would also allow other collections to opt-out of such things as well. 4. ""something really cool that I don't even know about"" Thoughts? " 1563 area-System.Drawing .NET core should have primitive drawing types Issue: #1363 Based on the analysis done by Experience and Insights, primitive drawing types are commonly used even outside Image processing scenarios. The following types are to be considered: - Point - Rectangle - Size From the discussion, it is clear we need int and float based versions of the primitives, we decided to expose the already existing System.Drawing primitives as is, that is, Point, PointF, Size, SizeF, Rectangle and RectangleF to .NET Core. We are not exposing Color at the moment because we feel the existing Color type in System.Drawing is insufficient as specified in the below conversations. Once the existing types are exposed, we will start a separate discussion on crafting Color primitive for .NET Core. **Contract** System.Drawing.Primitives **API Proposal** ``` csharp namespace System.Drawing { public struct Point { public static readonly Point Empty; public Point(Size sz); public Point(int dw); public Point(int x, int y); public bool IsEmpty { get; } public int X { get; set; } public int Y { get; set; } public static Point Add(Point pt, Size sz); public static Point Ceiling(PointF value); public override bool Equals(object obj); public override int GetHashCode(); public void Offset(Point p); public void Offset(int dx, int dy); public static Point operator +(Point pt, Size sz); public static bool operator ==(Point left, Point right); public static explicit operator Size (Point p); public static implicit operator PointF (Point p); public static bool operator !=(Point left, Point right); public static Point operator -(Point pt, Size sz); public static Point Round(PointF value); public static Point Subtract(Point pt, Size sz); public override string ToString(); public static Point Truncate(PointF value); } public struct PointF { public static readonly PointF Empty; public PointF(float x, float y); public bool IsEmpty { get; } public float X { get; set; } public float Y { get; set; } public static PointF Add(PointF pt, Size sz); public static PointF Add(PointF pt, SizeF sz); public override bool Equals(object obj); public override int GetHashCode(); public static PointF operator +(PointF pt, Size sz); public static PointF operator +(PointF pt, SizeF sz); public static bool operator ==(PointF left, PointF right); public static bool operator !=(PointF left, PointF right); public static PointF operator -(PointF pt, Size sz); public static PointF operator -(PointF pt, SizeF sz); public static PointF Subtract(PointF pt, Size sz); public static PointF Subtract(PointF pt, SizeF sz); public override string ToString(); } public struct Rectangle { public static readonly Rectangle Empty; public Rectangle(Point location, Size size); public Rectangle(int x, int y, int width, int height); public int Bottom { get; } public int Height { get; set; } public bool IsEmpty { get; } public int Left { get; } public Point Location { get; set; } public int Right { get; } public Size Size { get; set; } public int Top { get; } public int Width { get; set; } public int X { get; set; } public int Y { get; set; } public static Rectangle Ceiling(RectangleF value); public bool Contains(Point pt); public bool Contains(Rectangle rect); public bool Contains(int x, int y); public override bool Equals(object obj); public static Rectangle FromLTRB(int left, int top, int right, int bottom); public override int GetHashCode(); public static Rectangle Inflate(Rectangle rect, int x, int y); public void Inflate(Size size); public void Inflate(int width, int height); public void Intersect(Rectangle rect); public static Rectangle Intersect(Rectangle a, Rectangle b); public bool IntersectsWith(Rectangle rect); public void Offset(Point pos); public void Offset(int x, int y); public static bool operator ==(Rectangle left, Rectangle right); public static bool operator !=(Rectangle left, Rectangle right); public static Rectangle Round(RectangleF value); public override string ToString(); public static Rectangle Truncate(RectangleF value); public static Rectangle Union(Rectangle a, Rectangle b); } public struct RectangleF { public static readonly RectangleF Empty; public RectangleF(PointF location, SizeF size); public RectangleF(float x, float y, float width, float height); public float Bottom { get; } public float Height { get; set; } public bool IsEmpty { get; } public float Left { get; } public PointF Location { get; set; } public float Right { get; } public SizeF Size { get; set; } public float Top { get; } public float Width { get; set; } public float X { get; set; } public float Y { get; set; } public bool Contains(PointF pt); public bool Contains(RectangleF rect); public bool Contains(float x, float y); public override bool Equals(object obj); public static RectangleF FromLTRB(float left, float top, float right, float bottom); public override int GetHashCode(); public static RectangleF Inflate(RectangleF rect, float x, float y); public void Inflate(SizeF size); public void Inflate(float x, float y); public void Intersect(RectangleF rect); public static RectangleF Intersect(RectangleF a, RectangleF b); public bool IntersectsWith(RectangleF rect); public void Offset(PointF pos); public void Offset(float x, float y); public static bool operator ==(RectangleF left, RectangleF right); public static implicit operator RectangleF (Rectangle r); public static bool operator !=(RectangleF left, RectangleF right); public override string ToString(); public static RectangleF Union(RectangleF a, RectangleF b); } public struct Size { public static readonly Size Empty; public Size(Point pt); public Size(int width, int height); public int Height { get; set; } public bool IsEmpty { get; } public int Width { get; set; } public static Size Add(Size sz1, Size sz2); public static Size Ceiling(SizeF value); public override bool Equals(object obj); public override int GetHashCode(); public static Size operator +(Size sz1, Size sz2); public static bool operator ==(Size sz1, Size sz2); public static explicit operator Point (Size size); public static implicit operator SizeF (Size p); public static bool operator !=(Size sz1, Size sz2); public static Size operator -(Size sz1, Size sz2); public static Size Round(SizeF value); public static Size Subtract(Size sz1, Size sz2); public override string ToString(); public static Size Truncate(SizeF value); } public struct SizeF { public static readonly SizeF Empty; public SizeF(PointF pt); public SizeF(SizeF size); public SizeF(float width, float height); public float Height { get; set; } public bool IsEmpty { get; } public float Width { get; set; } public static SizeF Add(SizeF sz1, SizeF sz2); public override bool Equals(object obj); public override int GetHashCode(); public static SizeF operator +(SizeF sz1, SizeF sz2); public static bool operator ==(SizeF sz1, SizeF sz2); public static explicit operator PointF (SizeF size); public static bool operator !=(SizeF sz1, SizeF sz2); public static SizeF operator -(SizeF sz1, SizeF sz2); public static SizeF Subtract(SizeF sz1, SizeF sz2); public PointF ToPointF(); public Size ToSize(); public override string ToString(); } } ``` 1565 area-System.Runtime System.String should be consistent with taking StringComparison arguments Every method on `System.String` that compares strings should take a `StringComparison` argument. An example where that's not the case is, `String.Contains`. We should to a holistic pass to make sure we don't miss others. 1571 area-System.IO Add TextWriter.Write(string value, int index, int count) overload TextWriter has an overload for writing a segment of a char array with this Write overload: void Write(char[] buffer, int index, int count) But TextWriter doesn't have an overload for writing a segment of a string, e.g.: void Write(string value, int index, int count) This means that in cases where you have a segment of a string that you want to write you either need to either substring and pass that value to Write(string) or copy the string to a char array and then use the char array segement overload. It would be a new virtual method but the default implementation could simply call the current Write(char[] buffer, int index, int count) overload. The main problem I see with adding Write(string, int, int) is there is a Write(string, object, object) overload that does string formatting that would be hidden. Avoid with different name? Example of having to having to use the Write(char[] buffer, int index, int count) overload where a string overload would simplify code and improve performance: https://github.com/JamesNK/Newtonsoft.Json/blob/6a22345e28006d74c25e353b6235bc5222bab821/Src/Newtonsoft.Json/Utilities/JavaScriptUtils.cs#L220-L228 1577 area-System.Net HttpPrefixes So http prefixes ... Its an IIS specific API that makes zero sense when IIS is no longer present. I know this code hasn't come over yet code wise but I figured I would create an issue for it for discussion about what may happen. As an example I want to: Bind to 192.168.1.101 : 1111 Allow an elastic ip with a round robin dns to be used. In mono as example this will cause it to bind *:1111 and allow which is totally not what I wanted, there is no good way in the current api of actually specifying. I can specify 192.168.1.101:1111/ just fine. What happens when I as well specify myroundrobindns:1111/? Do I mean that I want to allow this host header or to bind to this interface? This is where the problem starts.... What if I only specified the myroundrobindns:1111? Then it would make sense to bind all or give me an error (without IIS). But this behaviour makes it impossible to do the above (common use case). There are literally dozens of derpy little gotchas like this (and no good way of working around them) due to how the api is laid out and not wanting to break compatibility with msclr. Whats the plan for CoreFX? Thoughts? Greg 1582 area-Infrastructure XML test libs included in coverage report I looked at the coverage report and noticed that a bunch of XML test libs are included: ![image](https://cloud.githubusercontent.com/assets/1376924/7392228/d392bf42-ee86-11e4-8254-db6d5ee5e20f.png) Those are dragging the overall coverage down and should be excluded from coverage. /cc @krwq 1583 area-System.Runtime Add generic overloads to Math.Min and Math.Max There are overloads of `Math.Min` and `Math.Max` for all the numeric types, but there aren't any for general comparable types (including framework types like `DateTime` and `TimeSpan`). It would be nice to have them. ## Rationale For integral types, `Math.Min` and `Math.Max` are not necessary, e.g. instead of `Math.Min(a, b)`, you can write `a <= b ? a : b`. But they are nice, because they make the meaning clear and decrease the likelihood of error. The same argument applies to general comparable types, so they should work with `Math.Min` and `Math.Max` too. ## Proposed API ``` c# public static class Math { public static T Min(T val1, T val2) where T : IComparable; public static T Min(T val1, T val2, IComparer comparer); public static T Max(T val1, T val2) where T : IComparable; public static T Max(T val1, T val2, IComparer comparer); } ``` ## Open questions - Should the constraint `where T : IComparable` be there? Other framework methods that require comparison, like `Enumerable.OrderBy`, don't have it and instead use `Comparer.Default`. - Should nullable value types be supported? The proposed version doesn't, because `T?` does not implement `IComparable`, even when `T` does implement `IComparable`. 1586 area-System.Net Initial open sourcing of System.Net.Http code. 1591 area-System.Reflection System.Reflection.Metadata.MetadataReader.MetadataReader consistently throws InvalidOperationException in store app I'm able to use System.Reflection.Metadata without any problems in my .NET 4.5.1 desktop app, but I'm unable to successfully use it in a Windows Store app--I consistently see this exception getting thrown: ``` System.InvalidOperationException occurred Message: Exception thrown: 'System.InvalidOperationException' in mscorlib.ni.dll Additional information: The API 'System.Text.Encoding.GetString(Byte*, Int32)' cannot be used on the current platform. See http://go.microsoft.com/fwlink/?LinkId=248273 for more information. ``` ...with this call stack: ``` mscorlib.ni.dll!System.Delegate.CreateDelegateInternal(System.RuntimeType rtType, System.Reflection.RuntimeMethodInfo rtMethod, object firstArgument, System.DelegateBindingFlags flags, ref System.Threading.StackCrawlMark stackMark) Unknown mscorlib.ni.dll!System.Reflection.RuntimeMethodInfo.CreateDelegateInternal(System.Type delegateType, object firstArgument, System.DelegateBindingFlags bindingFlags, ref System.Threading.StackCrawlMark stackMark) Unknown mscorlib.ni.dll!System.Reflection.RuntimeMethodInfo.CreateDelegate(System.Type delegateType, object target) Unknown > System.Reflection.Metadata.dll!System.Reflection.Internal.EncodingHelper.LoadGetStringPlatform() Unknown System.Reflection.Metadata.dll!System.Reflection.Internal.EncodingHelper.EncodingHelper() Unknown [Native to Managed Transition] [Managed to Native Transition] System.Reflection.Metadata.dll!System.Reflection.Internal.EncodingHelper.GetString(System.Text.Encoding encoding, byte* bytes, int byteCount) Unknown System.Reflection.Metadata.dll!System.Reflection.Metadata.MetadataStringDecoder.GetString(byte* bytes, int byteCount) Unknown System.Reflection.Metadata.dll!System.Reflection.Metadata.MetadataReader.ReadMetadataHeader(ref System.Reflection.Metadata.BlobReader memReader) Unknown System.Reflection.Metadata.dll!System.Reflection.Metadata.MetadataReader.MetadataReader(byte* metadata, int length, System.Reflection.Metadata.MetadataReaderOptions options, System.Reflection.Metadata.MetadataStringDecoder utf8Decoder) Unknown System.Reflection.Metadata.dll!System.Reflection.Metadata.PEReaderExtensions.GetMetadataReader(System.Reflection.PortableExecutable.PEReader peReader, System.Reflection.Metadata.MetadataReaderOptions options, System.Reflection.Metadata.MetadataStringDecoder utf8Decoder) Unknown ``` I see that the current implementation of `LoadGetStringPlatform` looks for `MemberAccessException`s as indication that some sort of platform-specific fallback is required; could it be that it needs to catch other exception types as well (i.e. `InvalidOperationException`)? 1593 area-System.Xml XPathNodeIterator should implement IEnumerable Currently XPathNodeIterator implements only IEnumerable interface. XPathNodeIterator always iterates over objects of type XPathNavigator and thus every usage ends up with casting back and forth XPathNavigator --> object --> XPathNavigator in a loop. Implementing IEnumerable on this type would allow to avoid this indirection and in cases where iterating is useful improve perf. cc: @tmat EDIT 2017/12/28: In PR: https://github.com/dotnet/corefx/pull/26083 Requested API changes: ```diff namespace System.Xml.XPath { ... - public abstract class XPathNodeIterator : ICloneable, IEnumerable + public abstract class XPathNodeIterator : ICloneable, IEnumerable ... ``` 1595 area-System.Xml Add XmlReader.ResetState to enable pooling (modified copy of internal bug / cc: @tmat) The XmlReader allocates char[] and byte[] buffers in its ParseState. XmlReader created via XmlReader.Create doesn't expose ResetState method that would allow pooling. XmlTextReader does, however the class is legacy and not available on new platforms (in portable profile). Or in simple words: put multiple xml files in one stream and try to parse it without creating multiple XmlReader instances which do a lot of allocations. Simple scenario is reading xml doc comments by Roslyn 1596 area-System.Globalization Randomized Hashing APIs Proposal is to add APIs which expose old hashing methods and the new one (old ones are non randomized). Background: On some platforms default is to use randomized hashing and on some it is non randomized. Exposing APIs allows devs to make it explicit so in case we want to always use randomized hashing for security reasons we can do it or if we want the hashing to be faster we can use non randomized. (Some work might need to be done in dotnet/coreclr) This was already reviewed internally once although here is another iteration. Proposal is to add following APIs ``` csharp //assembly System.Globalization { namespace System.Globalization { public class CompareInfo { public virtual int GetHashCode(string source, CompareOptions options, HashingOption hashingOption); } // enum numbers should be explicit in the final version public enum HashingOption { //Default, ??? NonRandomized, Randomized, } } //} //assembly System.Globalization.Extensions { namespace System.Globalization { public static class GlobalizationExtensions { public static StringComparer GetStringComparer(this CompareInfo compareInfo, CompareOptions options, HashingOption hashingOption); } } //} //assembly System.Runtime.Extensions { namespace System { public abstract class StringComparer : IComparer, IComparer, IEqualityComparer, IEqualityComparer { public static StringComparer OrdinalNonRandomized { get; } public static StringComparer OrdinalIgnoreCaseNonRandomized { get; } public static StringComparer OrdinalRandomized { get; } public static StringComparer OrdinalIgnoreCaseRandomized { get; } } } //} ``` 1597 area-Infrastructure Running CoreFx tests from VS is broken I’m using TestDriven.NET. 1) Right-click on a test method and selecting “Run Test” works well for System.Reflection.Metadata tests. Good. 2) Right-click on a test class or a test file reports an error in output: The target type doesn't contain tests from a known test framework or a 'Main' method. 3) Right-click on a test method and selecting “Test with … Debugger” freezes VS. Test Explorer doesn’t show any tests even after full solution build. Trying to set the test project as a startup project and F5 it. Error: “The working directory does not exist: 'C:\FX0\bin\tests\Windows_NT.AnyCPU.Debug\System.Reflection.Metadata.Tests\dnxcore50'.” Seems like building the entire CoreFx repo fixes this issue (although I’m not quite sure). In any case I don’t want to build the entire repo to run a single test. Couldn’t get F5 work from Dev12 VS, yet the project is Dev12 VS. With Dev14 I was able to debug the test. 1598 area-Meta Improve layout of bin\tests directory Building and running all CoreFx tests creates bin\tests directory with an insane amount of files (14493 files of total size 1.4GB). The problem is that every CoreFx library deploys its own copy of DNX and XUnit (175 files) to its test directory. Instead all tests should be deployed into a single directory with a single copy of DNX. 1603 area-Meta Add inner and outer loop testing explanation to Developer Guide 1605 area-System.Numerics Add Conversions methods to Vector It would be useful to support conversions between different types of Vector. The API and usage model is pretty straightforward for same-size types, e.g. Double and Int64, so I would suggest that we start by exposing that, and consider mixed-size converts as usage demands (we have had customer requests for at least the same-size convert). A possible instance method on Vector might look like this: public Vector ToVectorDouble(); and it would throw arg exception for non-64-bit vectors (or, alternatively, it could be defined as an extension method on the static Vector class for only those instantiations that are supported). I propose the following: - `Vector <=> Vector` - `Vector <=> Vector` - `Vector <=> Vector` - `Vector <=> Vector` 1608 area-System.Numerics Add Gather methods for Vector Vector Gathers are a commonly used paradigm in vector computation - where a gather involves constructing a vector from non-contiguous elements of a source array. Gather is supported on AVX2, but even on targets that do not directly support it, such a method would be useful, in order to avoid the overhead of having to use an array to hold the elements prior to contructing the Vector. VectorGather should perhaps be declared on the static Vector class, something like: `public static Vector VectorGather(T[] source, Vector indexVector) where T,U : struct` Initially, I think it would be reasonable to support only same/size convert and gather – but eventually we may want to consider mixed size (e.g. a gather to a Vector using the lower values of a Vector. AVX2 has a pretty rich mixture of these. @mellinoe what do you think? 1609 area-System.Diagnostics System.Diagnostics.Debug.Tests does not build on Linux System.Diagnostics.Debug.Tests currently fails trying to resolve types against System.Runtime. It is disabled in dir.props. 1611 area-System.Net HttpResponseHeader should have a Cookies property Hello, The HttpResponseHeader class (found at https://github.com/dotnet/corefx/blob/master/src/System.Net.Http/src/System/Net/Headers/HttpResponseHeaders.cs ) doesn't seem to include a standard and pretty trivial header, the Cookies header. Furthermore, as suggested by http://www.asp.net/web-api/overview/advanced/http-cookies, we should use the HttpResponseHeadersExtensions, found at https://aspnetwebstack.codeplex.com/SourceControl/latest#src/System.Net.Http.Formatting/HttpResponseHeadersExtensions.cs , however, as you can see in the code (and in the CookieHeaderValue class) - there's no way to configure default values, e.g. HttpOnly or Secured. In a regular ASP.NET website we can set the default values in web.config, but since ASP.NET WebAPI doesn't rely on that, there's no way, even in the WebAPI configuration registering method to set such values. Part from creating my own extension to add cookies with default values, I would suggest adding support for cookies in the HttpResponseHeaders class and having the option to set default values in registration time. Thank you very much. 1612 area-System.Data System.Data.SqlClient.SqlParameter Precision and Scale do not work via the abstract API "Package from dnxcore50 via ``` ""System.Data.SqlClient"": ""4.0.0-beta-*"", ``` This works: ``` SqlParameter p = ... p.Precision = 10; p.Scale = 5; ``` However, if instead we use the abstract base class (like most tools will be): ``` SqlParameter p = ... DbParameter d = p; d.Precision = 10; d.Scale = 5; ``` then all of `p.Precision`, `p.Scale`, `d.Precision`, `d.Scale` **remain at 0**, and consequently do not work correctly. The abstract API's `Scale` and `Precision` (and any other associated members) should mirror the concrete type's equivalent. This code is not in corefx yet, so I cannot fix it :( " 1613 area-System.Data System.Data.SqlClient does not allow enum parameters values (it does in .NET) "Is this an expected break? For example: ``` using (var cmd = connection.CreateCommand()) { cmd.CommandText = ""select @foo""; cmd.Parameters.AddWithValue(""@foo"", AnEnum.B); object value = cmd.ExecuteScalar(); AnEnum val = (AnEnum)value; // cheeky unbox behavior val.IsEqualTo(AnEnum.B); } enum AnEnum { A = 2, B = 1 } ``` In .NET this works fine; in CoreCLR this breaks on the `ExecuteScalar()` call, reporting an `ArgumentException` with text: > No mapping exists from object type AnEnum to a known managed provider native type. " 1616 area-System.Numerics bug: Vector extracting components with a literal leads to incorrect code gen. It appears that if you try to retrieve components of a `Vector` with a constant something wrong is occurring and an incorrect code is generated. This does not happen when extracting to an array, or when you use a variable (like say through a for-loop). This only happens when you extract literal components. Here is a simple reproduction. This happened in VS2015 RC ``` csharp static bool TestLiterals(Vector vl) { var v = vl[0]; var v1 = vl[1]; var v2 = vl[2]; var v3 = vl[3]; return v != 0 || v1 != 0 || v2 != 0 || v3 != 0; } static bool TestLoop(Vector vl) { for(int i=0; i < Vector.Count; i++) { var v = vl[i]; if (v != 0) return true; } return false; } static void Main(string[] args) { var array = new long[4] { 0, 0, 0, 1 }; var vector = new Vector(array); Console.WriteLine(Vector.Count); Console.WriteLine(TestLiterals(vector)); Console.WriteLine(TestLoop(vector)); } ``` On my i5 broadwell chip it printed: ``` 4 False True ``` This is very much the wrong result. The correct answer should be: ``` 4 True True ``` It's pretty easy to see that both methods should yield the same result. Here is the assembly of the `testliteral` And the error becomes obvious. ``` **00007FFB1E086E20 vmovupd ymm0,ymmword ptr [rcx] 00007FFB1E086E25 vmovd rax,xmm0 ** **00007FFB1E086E2A vmovupd ymm0,ymmword ptr [rcx] 00007FFB1E086E2F vmovaps ymm1,ymm0 00007FFB1E086E34 vpsrldq ymm1,ymm1,8 00007FFB1E086E3A vmovd rdx,xmm1 ** 00007FFB1E086E3F vmovupd ymm0,ymmword ptr [rcx] 00007FFB1E086E44 vmovaps ymm1,ymm0 00007FFB1E086E49 vpsrldq ymm1,ymm1,10h 00007FFB1E086E4F vmovd r8,xmm1 00007FFB1E086E54 vmovupd ymm0,ymmword ptr [rcx] 00007FFB1E086E59 vmovaps ymm1,ymm0 00007FFB1E086E5E vpsrldq ymm1,ymm1,18h 00007FFB1E086E64 vmovd rcx,xmm1 00007FFB1E086E69 or rax,rdx 00007FFB1E086E6C or rax,r8 00007FFB1E086E6F test rax,rax 00007FFB1E086E72 jne 00007FFB1E086E7E ``` The input vector is in `ymm0` and it is using `ymm1` as its scratch buffer, first copying and then shifting the register by the offset but is trying to pull the component out of `xmm1`! It appears that the compiler while having some concept of AVX registers, is still confused. 1617 area-System.Numerics Vector should allow variable length based size of available registers. Currently, it seems that `Vector` will always be the size of the largest available register, and has no support for using a smaller register. E.g. if you have AVX2, you currently can only build `Vector` of 256bit. For some algorithms it can be very useful to be able to use either a vector of size 16 or a vector of size 32. 1622 area-Infrastructure Consider open sourcing ApiCompat. Internally we have some tooling to ensure that implementation assemblies are compatible with contracts. It would be a good idea to open source that and use it as part of the build or CI system or something. I'm not sure if this will happen naturally when we actually start generating contracts in the open or if we'll need to take extra steps. 1624 area-System.Net An simple way to mock an httpClient.GetAsync(..) method for unit tests? `System.Net.Http` has now been uploaded to the repo :smile: :tada: :balloon: Whenever I've used this in some service, it works great but makes it hard to unit test => my unit tests don't want to actually ever hit that real end point. Ages ago, I asked @davidfowl what should we do? I hoping I paraphrase and don't misquote him - but he suggested that I need to fake up a message handler (ie. `HttpClientHandler`), wire that up, etc. As such, I ended up making a helper library called [HttpClient.Helpers](https://github.com/PureKrome/HttpClient.Helpers) to help me run my unit tests. So this works ... but it feels _very_ messy and .. complicated. I'm sure I'm not the first person that needs to make sure my unit tests don't do a real call to an external service. Is there an easier way? Like .. can't we just have an `IHttpClient` interface and we can inject that into our service? 1625 area-System.Runtime "[public api] System.Runtime.Environment - OSName(""FreeBSD"") or OSName(""BSD"")" "This issue is to discuss what the final identifier will be within the entire public API surface and what will be the build target identifier that msbuild will use. https://github.com/dotnet/corefx/issues/1576 started the discussion of wether to use `OSName(""FreeBSD"")` vs `OSName(""BSD"")` in the unit tests and is recommended reading before progressing with reading this issue. `FreeBSD` won but it can be easily refactored due to being a non-public contract. The BSD's (FreeBSD/NetBSD/OpenBSD/DragonFlyBSD) unlike Linux all have different kernels so potentially it might be required to split them out as separate platforms. This might be completely unnecessary whereby the additional BSD's may just work as a direct result of the port team's efforts. There are enough differences between the different BSDs that it could be needed to differentiate between them however. We have some time before a decision needs to be cemented in as there is a more pressing issue as mentioned by @stephentoub and registered as issue #1626 > There's what's likely a more pressing issue, which is actually building for FreeBSD. Today, when we need to specialize an assembly for a particular platform, we effectively have three builds, producing three different managed assemblies: Windows, Linux, OSX. Sounds like at least for now we'll need a fourth, FreeBSD. I suggest you start by modifying the build to support an IsFreeBSD property (or just IsBSD of you think there's a high chance that the implementations across BSDs will be the same even with varied kernels) along with the appropriate OSGroup targets. That can then be used in the csproj files as needed to specialize an assembly with FreeBSD-specific code. I feel uneasy about using either `OSName(""FreeBSD"")` (platform fragmentation) or `OSName(""BSD"")` (too generic) with the amount of information available at this point in time but obviously a direction needs to be chosen. @janhenke @josteink (core members of the portteam) have put forward on multiple times that they would prefer `FreeBSD` and I have no strong preferences in any which direction as long as the _future_ implications are considered carefully because this decision affects the public API surface. Ultimately advice, direction and a decision will be needed by MSFT on this matter. note: issue was edited/refactored based on feedback from josenink since original post. " 1626 area-Meta Support for FreeBSD ## Updated proposal from 2017/9 Proposal (by @karelz - https://github.com/dotnet/corefx/issues/1626#issuecomment-329840518) will be updated in top-post based on further discussion and proposal changes. We discussed community-driven port for FreeBSD with @RussellHaley (from FreeBSD community) and @wfurt (from .NET Core team) who both expressed interest in the work. Here's a plan proposal we put together (feedback / suggestions are welcome): 1. Produce binaries in CoreCLR & CoreFX repo targeting FreeBSD - using hacks is fine * Hard to parallelize, @wfurt will work on that * The build can be mix of builds from other platforms (Mac, Linux) targeting FreeBSD * We will need documented steps (on FreeBSD wiki) to reproduce the build with FreeBSD-specific bug fixes 2. Run & stabilize CoreCLR tests (using corerun) * Tests may be built on another platform * Goal: Provides basic quality of runtime 3. Run & stabilize CoreFX tests (using corerun) * Tests may be built on another platform * Note this requires xunit. We believe, based on our past porting experience, once [2] is done, xunit will just work. * This can be in theory parallelized with [2] - it may require shortcutting xunit (e.g. generate static execution recipe on another platform) * We can expose new OSPlatform API for FreeBSD when the pass rate is reasonable: see #23989 4. Full stack build on FreeBSD (using corerun as bootstrapper from [1]-[3]) * We will need all tools (nuget, msbuild, roslyn) to work on boostrapping .NET Core 5. Installers (FreeBSD ports) * First-stage: Using product binaries from nuget feeds * Second-stage: Build product from source (blocked on build from source effort) * Requires FreeBSD community expertise and guidance on design * Note: We can link FreeBSD packages also from official .NET Core download pages as community-support packages 6. Regular build and test runs on FreeBSD * Goal: Make sure changes in .NET Core repos breaking FreeBSD are known early * Design needed * Requires FreeBSD community expertise and guidance on design Operation principles: * Changes in [2]-[4] should be done primarily in CoreCLR/CoreFX repos (due to CLA signing requirements, code reviews from .NET Core team experts/members. etc.) * We will track high-level work on this issue. Specific bugs will be filed as separate issues. If anyone is interested in helping, please let us know here. We can easily distribute work items from [2] & [3] above once we are far enough with [1]. --------------------------------------------- ## Original proposal from @ghuntley from 2015/5 This issue is to discuss unit(s) of work to actually produce FreeBSD assemblies for corefx. > @stephentoub - There's what's likely a more pressing issue, which is actually building for FreeBSD. Today, when we need to specialize an assembly for a particular platform, we effectively have three builds, producing three different managed assemblies: Windows, Linux, OSX. Sounds like at least for now we'll need a fourth, FreeBSD. I suggest you start by modifying the build to support an IsFreeBSD property (or just IsBSD of you think there's a high chance that the implementations across BSDs will be the same even with varied kernels) along with the appropriate OSGroup targets. That can then be used in the csproj files as needed to specialize an assembly with FreeBSD-specific code. Related issue(s) - #1625 (OSGroup identifier in the public API) - #1576 (OSGroup identifier in the private API) /cc: @janhenke @josteink 1633 area-System.Reflection More offsets in MetadataReader I believe MetadataReader should have more APIs which tell me about offsets. I. e.: Let's say I'm writing strong name signer, there are few steps required (assuming that assembly is already delayed signed): - calculate hash, - sign hash, - inject signature to calculate hash you need to skip some parts of the PE file like: - strong name signature blob (including the 8 bytes in the header) - certificates blob (including 8 bytes header) - checksum so I need to have offsets and sizes for each of those. For blobs there are already APIs: i.e.: ``` peReader.PEHeaders.CorHeader.StrongNameSignatureDirectory.Size peReader.PEHeaders.CorHeader.StrongNameSignatureDirectory.RelativeVirtualAddress peReader.PEHeaders.PEHeader.CertificateTableDirectory.Size peReader.PEHeaders.PEHeader.CertificateTableDirectory.RelativeVirtualAddress ``` what is missing is: - checksum offset (and optionally a size) - blob headers for blob headers we can do simple change in System.Reflection.PortableExecutable.DirectoryEntry which is: ``` diff public struct DirectoryEntry { //... + public readonly int HeaderOffset; //... internal DirectoryEntry(ref PEBinaryReader reader) { + HeaderOffset = reader.CurrentOffset; RelativeVirtualAddress = reader.ReadInt32(); Size = reader.ReadInt32(); } //... } ``` For stuff like CheckSum offset we need something like this: ``` diff public sealed class PEHeader { //... + public int BaseOffset; // this might be always 0 so const would be fine too //... + public const int CheckSumOffset; + public const int CheckSumSize; + //etc. for other fields } ``` cc: @nguerrera @tmat What are your thoughts? With good modifications we could probably reuse some of those types when writing MetadataWriter (assuming we are planning to do that) 1635 area-System.Security System.Security.Cryptography.Hashing.Algorithms cannot resolve System.Text.Encoding and System.Threading.Tasks ? "Using beta5 TFMs : dnx451 and dnxcore50 and this is my project.json dependencies: ``` ""dependencies"": { ""BioBook.Data"": ""1.0.0-*"", ""HtmlMinifier.MiddleWare"": ""1.0.0-beta5-1000"", ""Microsoft.AspNet.Authentication"": ""1.0.0-beta5-*"", ""Microsoft.AspNet.Authentication.Cookies"": ""1.0.0-beta5-*"", ""Microsoft.AspNet.Diagnostics"": ""1.0.0-beta5-*"", ""Microsoft.AspNet.Diagnostics.Entity"": ""7.0.0-beta5-*"", ""Microsoft.AspNet.Hosting"": ""1.0.0-beta5-*"", ""Microsoft.AspNet.Identity.EntityFramework"": ""3.0.0-beta5-*"", ""Microsoft.AspNet.Mvc"": ""6.0.0-beta5-*"", ""Microsoft.AspNet.Mvc.TagHelpers"": ""6.0.0-beta5-*"", ""Microsoft.AspNet.Server.IIS"": ""1.0.0-beta5-*"", ""Microsoft.AspNet.Server.WebListener"": ""1.0.0-beta5-*"", ""Microsoft.AspNet.StaticFiles"": ""1.0.0-beta5-*"", ""Microsoft.AspNet.Tooling.Razor"": ""1.0.0-beta5-*"", ""Microsoft.DataAnnotations"": ""1.0.0-beta1-*"", ""Microsoft.Framework.CodeGenerators.Mvc"": ""1.0.0-beta5-*"", ""Microsoft.Framework.ConfigurationModel.Json"": ""1.0.0-beta5-*"", ""Microsoft.Framework.DependencyInjection"": ""1.0.0-beta5-*"" }, ""frameworks"": { ""dnx451"": { ""frameworkAssemblies"": { } }, ""dnxcore50"": { ""dependencies"": { } } }, ``` When building it returns 2 errors ``` Error Dependency System.Text.Encoding >= 4.0.0-beta-22830 could not be resolved Error Dependency System.Threading.Tasks >= 4.0.0-beta-22830 could not be resolved ``` If I remove Microsoft.AspNet.Mvc.TagHelpers, the error goes away. Looking at dnu list --details I found that tag helpers doesn't have a dependency for those packages but System.Security.Cryptography.Hashing.Algorithms looks like it has. (it think that package is not open source yet, because a) can't find it. b) https://github.com/dotnet/corefx/issues/836 ) Also, removing Microsoft.AspNet.Mvc.TagHelpers but adding System.Security.Cryptography.Hashing.Algorithms triggers the same errors. If I explicit add those packages as dependencies the errors goes away like this: ``` ""dependencies"": { ""System.Text.Encoding"": ""4.0.10-beta-22830"", ""System.Threading.Tasks"": ""4.0.10-beta-22830"", ... }, ``` **Note:** I was asked on jabbr.net if I had a Compiler/Preprocess folder with uncommented directives. I do have that folder with one file `RazorPreCompilation.cs` but the code inside that file is commented. Also removing the folder does not fixes the build errors. Not sure what else I can do/look at but if you need anything else just let me know. " 1641 area-Infrastructure corefx fails on Arch Linux: Not registered task DownloadFile "I'm not sure whether corefx is supposed to work on Linux at this point, and I have no clue how to build this anyway. Still running `xbuild` in the top directory produces this. ``` [hritcu@detained corefx]$ xbuild XBuild Engine Version 12.0 Mono, Version 3.12.1.0 Copyright (C) 2005-2013 Various Mono authors Build started 5/6/2015 2:42:54 PM. __________________________________________________ Project ""/home/hritcu/Temp/corefx/build.proj"" (default target(s)): Target _RestoreBuildTools: Restoring build tools... : error : Error initializing task DownloadFile: Not registered task DownloadFile. Build FAILED. Errors: /home/hritcu/Temp/corefx/build.proj (default targets) -> /home/hritcu/Temp/corefx/dir.targets (_RestoreBuildTools target) -> : error : Error initializing task DownloadFile: Not registered task DownloadFile. 0 Warning(s) 1 Error(s) Time Elapsed 00:00:00.0923120 ``` " 1642 area-System.Numerics [BigInteger] Parse test with trailing spaces fails on some cultures ``` System.Numerics.Tests.parseTest.RunParseToStringTests [FAIL] Assert.Throws() Failure Expected: typeof(System.FormatException) Actual: (No exception was thrown) Stack Trace: m:\Workspace\github\corefx\src\System.Runtime.Numerics\tests\BigInteger\parse.cs(408,0): at System.Numerics.Tests.parseTest.VerifyParseToString(String num1, NumberStyles ns, Boolean failureNotExpected, String expected) m:\Workspace\github\corefx\src\System.Runtime.Numerics\tests\BigInteger\parse.cs(380,0): at System.Numerics.Tests.parseTest.VerifyParseToString(String num1, NumberStyles ns, Boolean failureNotExpected) m:\Workspace\github\corefx\src\System.Runtime.Numerics\tests\BigInteger\parse.cs(247,0): at System.Numerics.Tests.parseTest.VerifyNumberStyles(NumberStyles ns, Random random) m:\Workspace\github\corefx\src\System.Runtime.Numerics\tests\BigInteger\parse.cs(33,0): at System.Numerics.Tests.parseTest.RunParseToStringTests() Finished: System.Runtime.Numerics.Tests === TEST EXECUTION SUMMARY === System.Runtime.Numerics.Tests Total: 2, Errors: 0, Failed: 1, Skipped: 0, Time: 20,751s ``` Repro steps: 1. Start->Run (WinKey + R) -> intl.cpl 2. Change format (no need to change locale) to Ukrainian (Ukraine) 3. `build /p:WithCategories=OuterLoop` **Possible solution:** https://github.com/dotnet/corefx/pull/1592#issuecomment-99520215 1645 area-System.IO Add option on StreamWriter not to emit Byte Order Mark (BOM) Someone posted on CoreClr: https://github.com/dotnet/coreclr/issues/933 a requirement to have StreamWriter to optionally not emit BOM to the stream. A possible solution is to either add a defaulted bool parameter to each/some of the constructors (bool emitBOM = True), or add a new enum e.g. StreamWriterOptions with EmitBOM as one of the enums and again having that enum as a defaulted parameter on each/some of the constructors. There maybe an issue with binary compatibility so it might be better to add a new constructor with the bool/enum.' Comments? 1647 area-System.Text Expose Hex/Char utilities from System.Text.Encoders.Web as public APIs (on some type) See: src/System.Text.Encodings.Web/src/System/Text/Encodings/Web/HexUtil.cs 1650 area-System.Security Investigate handle usage in src/common/src/Interop/Windows/secur32/Interop.LSAStructs.cs This issue came up but was not addressed in the initial commit of System.Security.Principal and System.Security.Principal.WIndows. See the constructor in UNICODE_INTPTR_STRING. The comment was regarding whether we need to add DangerousAddRef / DangerousReleasing around DangerousGetHandle. 1652 area-System.Security Address possible race conditions in System.Security.Principal.Windows tests "See src/system.security.principal.Windows/tests/WindowsIdentityTests.cs Initial commit feedback was: ""Could this not be problematic / prone to race conditions if the safe handle gets finalized concurrent with usage of logonToken (no AddRef/Releasing is being done to prevent that)?"" " 1653 area-System.IO Named pipes and array segment Hello. was thinking why the named pipe api doesn't provide the possibility to use array segments as the socket api to be able to use buffer pooling to reduce the impact of pinning for arrays that are not big enough to make it to LOH. is a common technique with socket to avoid possible out of memory exception due to fragmentation. I am having nightmares because a new app we are building is doing a lot of io via named pipes and now OOM exceptions are all over the place. Any plan about this kind of features? 1657 area-System.IO FileSystemWatcher_4000_Tests.FileSystemWatcher_IncludeSubDirectories test failing in CI This test has failed multiple times in the last few days: ``` MESSAGE: Didn't observe a created event within 500ms +++++++++++++++++++ STACK TRACE: at Utility.ExpectEvent(WaitHandle eventOccured, String eventName, Int32 timeout) in d:\j\workspace\dotnet_corefx_windows_release\src\System.IO.FileSystem.Watcher\tests\Utility\Utility.cs:line 94 at FileSystemWatcher_4000_Tests.FileSystemWatcher_IncludeSubDirectories_Directory() in d:\j\workspace\dotnet_corefx_windows_release\src\System.IO.FileSystem.Watcher\tests\FileSystemWatcher.IncludeSubDirectories.cs:line 67 ``` 1664 area-System.Reflection System.Reflection.Metadata fails to read offset of DataDirectories spanning across multiple sections "If an assembly contains any DataDirectory which is in more than one section we throw ""System.BadImageFormatException : Section too small."". While I agree we should not have any assemblies like that we unfortunately already shipped one which has this issue. I believe we can still calculate the offset without checking ranges (or we can check the global range only) Example can be found here: https://github.com/krwq/corefx/tree/pereaderdatadir This is basically: ``` csharp using (FileStream fs = new FileStream(@""C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.JScript.dll"", FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete)) { using (PEReader reader = new PEReader(fs, PEStreamOptions.LeaveOpen | PEStreamOptions.PrefetchEntireImage | PEStreamOptions.PrefetchMetadata)) { int offset; if (reader.PEHeaders.TryGetDirectoryOffset(reader.PEHeaders.PEHeader.CertificateTableDirectory, out offset)) { } } } ``` " 1665 area-System.Reflection Vague/invalid Strong Name Signing description in ECMA-335 ECMA-335 description of strong name signing (II.6.2.1.3): - To calculate the hash, the signature is zeroed, the hash calculated, and then the result is stored into the signature. - The Strong Name (SN) signing process uses standard hash and cipher algorithms for Strong name signing. An SHA-1 hash over most of the PE file is generated. That hash value is RSA-signed with the SN private key. For verification purposes the public key is stored into the PE file as well as the signed hash value. - Except for the following, all portions of the PE File are hashed: - The Authenticode Signature entry: PE files can be authenticode signed. The authenticode signature is contained in the 8-byte entry at offset 128 of the PE Header Data Directory (“Certificate Table” in §II.25.2.3.3) and the contents of the PE File in the range specified by this directory entry. [Note: In a PE File conforming to this specification, this entry shall be zero. end note] - The Strong Name Blob: The 8-byte entry at offset 32 of the CLI Header (“StrongNameSignature” in §II.25.3.3) and the contents of the hash data contained at this RVA in the PE File. If the 8-byte entry is 0, there is no associated strong name signature. - The PE Header Checksum: The 4-byte entry at offset 64 of the PE Header Windows NT-Specific Fields (“File Checksum” in §II.25.2.3.2). [Note: In a PE File conforming to this specification, this entry shall be zero. end note] What we actually do is (hashing): - In CorHeader/CLI Header set StrongNameSigned flag (value: 8). (I believe this is starting on 12th byte of CLI Header and is UInt32 - do not remove the old flag value just do bitwise OR with value 8) - Set Checksum (4 bytes) to zeros (this is in optional PE header in NT specific fields) - Set Certificate Directory entry (only 8 byte header) to zeros - Calculate hash of PE header from the start until sections headers are finished. Notes: - Continue hashing each section in the order of occurrence in the headers (this will start immediately after section headers and will intentionally skip padding after section headers). Note: - Strong Name Directory - entirely skip from hashing (but do hash its data directory 8 byte header as is) After we get a hash we sign it and store in Strong Name Directory 1683 area-Infrastructure System.Collections.Tests runs against Nuget package not compiled output "Unit tests in System.Collections.Tests.csproj appear to execute against the System.Collections package, and not the compiled output in the bin folder. Any local changes made to System.Collections are not picked up by System.Collections.Tests during compilation at command line (build.bat, msbuild System.Collections.sln) or in Visual Studio. For example, new types are not found by the test project. Compiling in verbose mode shows that the reference to System.Collections.dll is being pulled from the packages folder (packages/System.Collections/4.0.10-beta/...) and not the compiled output of the local build. Chicken and the egg scenario, where compiling the solution for System.Collections depends on it having been successfully compiled (in the MS build lab)! In addition, some of the behavior appears to be intentional. In System.Collections.Tests.csproj, a custom comment was added above the : ""Compile tests against the contract, but copy our local-built implementation for testing"". The behavior makes it impossible to write unit tests for new logic (e.g. in forks) for System.Collection.dll. " 1690 area-Meta Publish document about reference assemblies, contracts, and facades. We are beginning to open source our facades and soon our contracts and so we need to add some documentation for people to better understand how these all work. 1691 area-System.Linq [Compiler Bug] PropertyInfo retrieved from LambdaExpression reports wrong ReflectedType when property is derived from another class This is an old bug: https://connect.microsoft.com/VisualStudio/feedback/details/554853 https://connect.microsoft.com/VisualStudio/feedback/details/607165/propertyinfo-retrieved-from-lambdaexpression-reports-wrong-reflectedtype-when-property-is-derived-from-another-class 1712 area-System.Console Investigate if we still need all the thread jumping machinery in Console's ControlCHandler Console's ControlCHandler tries to queue work the threadpool to actually call the user code which handles the control c event. This was added as part of runtime hardening during the 2.0 days. The original fix for the issue (which introduced this jumping into another thread goo) had this comment: > On 64-bit platforms the ControlC handler is executed on a thread created > outside the runtime with very small stack. Our SO checking now > conservatively checks for space, which leaves insufficient space to > allow an exception to be thrown and caught. > > The change is to launch the user event on a ThreadPool thread instead of > in the ControlC handler thread. Care is taken to ensure that the thread > does not deadlock, even if we fail to successfully begin execution of > the Threadpool thread. @jkotas wonders if this is needed anymore: > Looking at the history, this code was added in Whidbey days where we were trying to harden the runtime against stack overflows by adding probes for large amounts of stack everywhere. We have given up on it since then. Internally we have moved the code to use Tasks instead of the ThreadPool directly (as part of some general work around some internal testing scenarios for .NET Native), but we might just be able to get out of the game completely. 1718 area-System.Reflection Merges changes from master to dev/metadata 1728 area-System.IO Determine whether to change Unix behavior when getting time from a non-existent file On Windows, for back-compat reasons, getting CreationTime (and maybe last access/write time, too) from a non-existent file via FileInfo returns the equivalent of DateTime.FromFileTime(0), rather than throwing an exception. On Unix, we do what's arguably the more expected behavior and throw. We should determine whether we want to change the Unix implementation to match what's there on Windows. 1732 area-System.Reflection Update package revision number to 00003 1739 area-System.Security Interop\Windows\BCrypt\Cng.cs does not follow the Interop guidelines Cng.cs was ported as-was from closed source into open. It should be broken up into multiple Interop.Method.cs and/or Interop.MethodCollection.cs files, and the SafeHandle types extracted into files of their own right. 1742 area-Infrastructure Coverage report should consider OuterLoop tests I have a reason to believe that coverage report doesn't count OuterLoop tests. When looking at: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_coverage_windows/lastStableBuild/Code_Coverage_Report/ I can see test project (I know it shouldn't be on report): System.Xml.RW.XmlReaderLib has 0% coverage which is impossible as I'm 100% sure some outer-loop only tests projects are including it and run tests from there (and thus test the test project). Also ridiculously low Xml.ReaderWriter coverage seems to be confirming it (I know coverage isn't perfect but I'm pretty sure it is much more than 17%). 1745 area-System.Net URI class throws with Unix file paths "The following is thrown from System.Uri.CreateThis(...) when you are using Unix file paths to construct a URI object. ""System.UriFormatException: Invalid URI: The format of the URI could not be determined."" I know for windows there is special logic to add file:// to the front of the path, and Mono handles the paths correctly, but using CoreCLR non-windows it throws. " 1751 area-System.Net "Overzealous ""URI not absolute"" InvalidOperationExceptions from Uri instance members" "Many of Uri's instance properties throw an InvalidOperationException complaining that the operation is not supported for a relative URI. In some cases, this doesn't really make sense. The best example I can think of is the Uri.HostNameType property. Consider the scenario where one constructs an instance of the Uri class with the path ""www.domain.com"". As expected, this evaluates to a relative path, because the protocol has not been specified. However, absolute or not, one would expect the value of its Uri.HostNameType to be UriHostNameType.Dns. Instead, checking the HostNameType throws an exception. This particular scenario also exhibits an inconsistency with other members of the Uri class. Uri.CheckHostName(string name) is a static method which returns a UriHostNameType value, just the same as the Uri.HostNameType instance property. One could reasonably expect the following two calls to return the same value: var hostType = new Uri(""www.domain.com"", UriKind.RelativeOrAbsolute).HostNameType; var hostType = Uri.CheckHostName(""www.domain.com""); Unfortunately, the first call throws an exception. The second call using CheckHostName, however, returns the expected value: UriHostNameType.Dns. " 1756 area-System.Text Implement generic interfaces on Regex collections Implement `IList`, `IReadOnlyList`, and `IList` on the Regex collections. Fixes #271 1761 area-Serialization Remove dependency from System.Xml.XmlSerializer on System.Runtime.WindowsRuntime 4.0.10 "I had to workaround an issue in System.Xml.XmlSerializer with the current packages: System.Runtime.WindowsRuntime 4.0.0 has a depedendency on ""windows"" package which doesn't exist. This was caused by our auto-generation task not filtering the winrt reference in the reference assembly's surface area. I've fixed this in a separate change and we can remove the workaround once we have a new set of packages next week. " 1768 area-Serialization XmlSerializer Tests Failing after latest merge from TFS 1774 area-System.Security System.Security.Cryptography.Hashing.Algorithms: HMAC::get_Key behavior differs from desktop on large keys. "Desktop: ``` HMAC hmac = new HMACSHA1(); hmac.Key = new byte[65]; int length = hmac.Key.Length; // 20 ``` CoreFX: ``` HMAC hmac = new HMACSHA1(); hmac.Key = new byte[65]; int length = hmac.Key.Length; // 65 ``` [RFC 2104, section 2](https://tools.ietf.org/html/rfc2104#section-2) says ""Applications that use keys longer than B bytes will first hash the key using H and then use the resultant L byte string as the actual key to HMAC."". In this case, SHA-1's block size (B) is 64, so the key is replaced with H(K). And then hmac.Key should return the H(K) answer, instead of the value which was set. Once this bug is fixed, there should be a test that verifies it. " 1776 area-Infrastructure Test runs are disabled when building on Linux See the skip of RunTestsForProject in https://github.com/dotnet/corefx/blob/master/mono.targets. 1784 area-System.Reflection Assembly.GetExecutingAssembly, AppDomain.CurrentDomain and similar Are there any plans to include the following classes and methods in near future? I am porting my framework (http://github.com/riganti/redwood) to .NET Core and I miss these methods. - Assembly.GetReferencedAssemblies - Assembly.GetExecutingAssembly - Assembly.GetEntryAssembly - Assembly.GetCallingAssembly - Assembly.CreateInstance - Assembly.Load which takes byte array or a stream - AppDomain.CurrentDomain.GetAssemblies - AppDomain.CurrentDomain.AssemblyResolve Or is there another way in the new .NET Core how to list all loaded assemblies, get the assembly of method that is being executed or a calling method etc.? 1786 area-System.Reflection Merge branch 'master' of https://github.com/dotnet/corefx into dev/metadata 1787 area-Infrastructure System.IO.Compression.ZipFile.Tests failing in microbuild of dev/metadata branch Disabling these tests in dev/metadata branch until these issues are resolved. ``` System.IO.Compression.Test.ZipTest.CreateFromDirectoryNormal System.IO.Compression.Test.ZipTest.CreateEntryFromFileTest System.IO.Compression.Test.ZipTest.UpdateReadTwice System.IO.Compression.Test.ZipTest.UpdateAddFile ``` Stack trace is the same for all failures: ``` System.IO.Compression.ZLibException : The underlying compression routine could not be loaded correctly. ---- System.DllNotFoundException : Unable to load DLL 'clrcompression.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E) Stack Trace: at System.IO.Compression.DeflaterZLib.DeflateInit(CompressionLevel compressionLevel, Int32 windowBits, Int32 memLevel, CompressionStrategy strategy) at System.IO.Compression.DeflaterZLib..ctor(CompressionLevel compressionLevel) at System.IO.Compression.DeflateStream.CreateDeflater(Nullable`1 compressionLevel) at System.IO.Compression.DeflateStream..ctor(Stream stream, CompressionMode mode, Boolean leaveOpen) at System.IO.Compression.ZipArchiveEntry.GetDataCompressor(Stream backingStream, Boolean leaveBackingStreamOpen, EventHandler onClose) at System.IO.Compression.ZipArchiveEntry.OpenInWriteMode() at System.IO.Compression.ZipArchiveEntry.Open() ``` 1788 area-System.Reflection Disable System.IO.Compression.ZipFile.Tests failing in microbuild of dev/metadata branch 1790 area-Meta .NET Core API inclusion principles doc So, I was responding in #1420 and tried to hunt down some docs here that had a set of principles for what was/should be included in .Net Core. I couldn't find anything in my quick search. Maybe I missed it? Maybe we need to get something better together? /cc @nexussays @nguerrera @blackdwarf 1791 area-Infrastructure GIT fails to sync frequently in internal corefx-dev-metadata-realsigned queue. "``` EXEC: src refspec dev/metadata does not match any. EXEC: failed to push some refs to 'https://github.com/dotnet/corefx.git' corefx-dev-metadata-realsigned\src\packages\Microsoft.DotNet.BuildTools.1.0.25-prerelease-00048\lib\UpdateBuildValues.targets (39): The command ""git push origin dev/metadata"" exited with code 1. ``` " 1793 area-System.IO Please add support for BufferedStream type in .NET Core We have a scenario where we implement our own stream and use the BufferedStream class to add a buffering layer on top of it. 1794 area-System.Net Why does HttpClient dispose HttpContent? Hi. I'm trying to write some tests on our custom protocol over http. So, I passed a fake message handler to client, like this (we don't use C#6 yet, so): ```c# public class CachingRequestsHandler : DelegatingHandler { public CachingRequestsHandler() { this.Requests = new List(); } public List Requests { get; private set; } protected override Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) { this.Requests.Add(request); return Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)); } } ``` When in test I'm trying to inspect request's `Content`, I'm getting an `ObjectDisposedException`. As I found `HttpClient` [unconditionally disposes](https://github.com/dotnet/corefx/blob/master/src/System.Net.Http/src/System/Net/HttpClient.cs#L360) `Content`. I have two questions: - will this behavior change in future to give user some control over Content? - how should I work around this issue, because I want to assert and test all parts of request: headers and content? Is there a fast and easy way to clone `HttpRequestMessage`? 1797 area-System.Diagnostics Implement System.Diagnostics.StackTrace/StackFrame Per discussion in #1420, this issue is to design & implement equivalents for [StackFrame](http://referencesource.microsoft.com/#mscorlib/system/diagnostics/stackframe.cs) and [StackTrace.](http://referencesource.microsoft.com/#mscorlib/system/diagnostics/stacktrace.cs) taking into account any possible API changes that might be necessary to better fit the goals of .NET Core. Implementation discussion may want to reference #1790 1805 area-System.Reflection Section name in PEReader contains null padding String in section name (i.e. peReader.PEHeaders.SectionHeaders[0].Name) contains null characters. I believe we should remove it, unless it is by design. ECMA description of field: ``` An 8-byte, null-padded ASCII string. There is no terminating null if the string is exactly eight characters long. ``` Here is how we get the value: ``` csharp // reader is PEBinaryReader // _name is string // PEFileConstants.SizeofSectionName = 8 _name = reader.ReadUTF8(PEFileConstants.SizeofSectionName); // impl of ReadUTF8: public string ReadUTF8(int byteCount) { byte[] bytes = ReadBytes(byteCount); return Encoding.UTF8.GetString(bytes, 0, byteCount); } ``` In theory it can also be encoding bug. 1813 area-System.Security Oid.FriendlyName gives different answers on Unix and Windows "``` new Oid(""1.2.840.113549.1.1.1"").FriendlyName ``` Windows: `""RSA""` Unix: `""rsaEncryption""` The tests for `FriendlyName` may encounter some errors when they get ported and CI tries to evaluate them. Or maybe there will be a sufficient core of sameness to allow the tests to keep running. " 1815 area-System.Reflection PEReader throws on an empty PE image 1820 area-System.Security Interop\Windows\Crypt32\OidInfo.cs does not follow the Interop guidelines OidInfo.cs was ported as-was from closed source into open. It should be broken up into multiple Interop.Method.cs and/or Interop.MethodCollection.cs files. If it is already a reasonable MethodCollection set, it still needs the type/namespace names adjusted, and the file to be renamed. 1821 area-System.Reflection Merge branch 'master' 1825 area-System.Net HttpClient ReadAsync waits for whole buffer to fill up even if HttpCompletionOption.ResponseHeadersRead is set Hi, I've prepared a simple project to demonstrate my issue: https://github.com/ppekrol/ReadAsyncIssue Steps to reproduce: 1. Run web (ReadAsyncIssue), can be under DNX451 or DNXCORE50, no difference 2. Run console app: a) when DNX451 is used, ReadAsync returns immediately (read length = 100) b) when DNXCORE50 is used, ReadAsync waits for buffer to fill (8192) I think this is a bug. 1831 area-Serialization Two XML Tests failing due to new PlatformNotSupportedException fc5506a2 introduced a platform not supported exception in a few APIs so the behavior could align with .NET Native. This is causing a few failing tests. 1832 area-System.Reflection Use immutable collections 1.1.36 in metadata 1836 area-System.Linq System.Linq.Parallel Exceptions not consistent The exceptions thrown (or wrapped) by PLINQ aren't consistent. Enumerables throwing an exception during `GetEnumerator()` will expose the original exception, and won't wrap it in `AggregateException` (which at least a naïve reading of the documentation suggests it should), for everything but the following 'output' methods: - `Average` - `Count` - `LongCount` - `Max` - `Min` - `Sum` ... and `SequenceEqual`, but only when some other method is called first (eg, `new FailingEnumerable.AsParallel().Select(x => x).SequenceEquals(b)`). If the enumerator itself throws an exception (from `MoveNext()` or `Current`), and additional PLINQ method is called first: 1. `SequenceEqual` throws nested `AggregateException`s 2. The following 'output' methods throw the original exception, and will not wrap it in an `AggregateException`: - `Aggregate(seed, accumulator)` - `Aggregate(seed, accumulator, resultSelector)` - `foreach(...)` - `ToArray` - `ToDictionary` - `ToList` This is, at minimum, a 'gotcha' waiting to trip somebody up. How should we handle it? Interestingly, `SequenceEqual` has: > // If first.GetEnumerator throws an exception, we don't want to wrap it with an AggregateException. > // If second.GetEnumerator throws an exception, we don't want to wrap it with an AggregateException. ... which I have a feeling was to deal with the case where an inner operator threw something already wrapped (to not nest it), but ended up missing the other cases.
I've got a [Gist](https://gist.github.com/Clockwork-Muse/c10553dbce3de937aecd) of the tests I'm using to find this info, although I wasn't planning on adding them to the PLINQ tests in this form. Especially because some stuff has been ad-hoc tweaked to be able to find the info better 1838 area-System.Diagnostics Provide way to send Control+C to proccess Currently .NET doesn't have way to exit nicely from created console process. It would be nice if we had a such a method. ``` csharp namespace System.Diagnostics { public class Process { public void SendCtrlCSignal() { ... } } } ``` It can be implemented following way on Windows: http://stackoverflow.com/a/15281070/61505 , for unix we can call `kill (pid, SIGINT);` http://stackoverflow.com/a/1761182/61505 It provide a lot of problems for other users too http://stanislavs.org/stopping-command-line-applications-programatically-with-ctrl-c-events-from-net/ 1839 area-System.Reflection Merge master and use immutable collections 1.1.36 in metadata 1840 area-System.IO System.IO.Pipes.Tests fail with EntryPointNotFoundException on Win10 I have disabled the following tests in dev/metadata branch until they are fixed. ``` AnonymousPipesSimpleTest.ClientPInvokeChecks [FAIL] System.EntryPointNotFoundException : Unable to find an entry point named 'GetNamedPipeHandleStateW' in DLL 'api-ms-win-core-namedpipe-l1-2-1.dll'. Stack Trace: at Interop.mincore.GetNamedPipeHandleState(SafePipeHandle hNamedPipe, Int32& lpState, IntPtr lpCurInstances, IntPtr lpMaxCollectionCount, IntPtr lpCollectDataTimeout, IntPtr lpUserName, Int32 nMaxUserNameSize) C:\FX0\src\System.IO.Pipes\src\System\IO\Pipes\PipeStream.Windows.cs(983,0): at System.IO.Pipes.PipeStream.UpdateReadMode() C:\FX0\src\System.IO.Pipes\src\System\IO\Pipes\PipeStream.Windows.cs(236,0): at System.IO.Pipes.PipeStream.get_ReadMode() C:\FX0\src\System.IO.Pipes\tests\AnonymousPipesSimpleTest.cs(221,0): at AnonymousPipesSimpleTest.ClientPInvokeChecks() NamedPipesSimpleTest.ClientServerMessages [FAIL] System.EntryPointNotFoundException : Unable to find an entry point named 'GetNamedPipeHandleStateW' in DLL 'api-ms-win-core-namedpipe-l1-2-1.dll'. Stack Trace: at Interop.mincore.GetNamedPipeHandleState(SafePipeHandle hNamedPipe, IntPtr lpState, IntPtr lpCurInstances, IntPtr lpMaxCollectionCount, IntPtr lpCollectDataTimeout, StringBuilder lpUserName, Int32 nMaxUserNameSize) C:\FX0\src\System.IO.Pipes\src\System\IO\Pipes\NamedPipeServerStream.Windows.cs(149,0): at System.IO.Pipes.NamedPipeServerStream.GetImpersonationUserName() C:\FX0\src\System.IO.Pipes\tests\NamedPipesSimpleTest.cs(389,0): at NamedPipesSimpleTest.ClientServerMessages() Finished: System.IO.Pipes.Tests ``` 1843 area-Serialization Workaround for perf issue with the new jit pointer dereference in serialization With the new jit, the *chars deference is done twice which is more costly compared to legacy jit 1844 area-System.Net Support System.Net.Security.NegotiateStream Add support for NegotiateStream that exists in the full .NET Framework. This involves either creating a new library or a newer version of an existing one. As a minimum it should support the same client API surface as the full .NET Framework. This is required for WCF to complete its NetTcpBinding implementation. 1849 area-System.IO Unix: Why not use Unix Domain Sockets for Named Pipes? System.IO.Pipes map to native windows implementations of anonymous and named pipes. Windows anonymous are close in implementation to Unix pipes in that they are one way and byte oriented (not datagram or message oriented). On the other hand, Named pipes can be either full (supporting I/O in both directions) or half (I/O in one direction) and messages can either be byte oriented or message/datagram oriented. Pipe connections can also be made across systems through the network (I believe using SMB) Unix has named pipes called FiFo's but they are one way only and not message oriented. The current Pipes port for corefx under unix uses FIfo's and therefore is a subset implementation. Instead of using Fifo's why don't we use Unix Domain Sockets? They support most of the functionality of windows named pipes including full/half duplex, byte or message oriented. They however don't support cross system connections. see overview of Unix Domain Sockets here: http://www.thomasstover.com/uds.html Ultimately, what is the goal of the corefx library on a different platform? To bind to existing similar functionality? or to provide strict portability of the api's across platforms (what I'm calling emulation)? If we strictly bind, then there will always be limitations and differences on functionality provided thus decreasing portability. However by binding, we are hooking into existing OS capabilities and can in theory hook up to other non-net apps on the host platform. If we strictly emulate (e.g. say grab a chunk of shared memory and then using semaphores to strictly emulate named pipes functionality under Unix), then we can be extremely portable however we can't communicate with anything else. A possible solution is to extend what can be used as a 'pipe name' in the system.io.pipes implementation where the syntax can be extended to give hints as to what to bind to in the underlying implementation but with the default (without any extended syntax) defaulting to an emulated functional solution. Comments? 1854 area-System.Net "SslStream.AuthenticateAsClientAsync() fails with ""The buffers supplied to a function was too small""" From time to time I receive this error message. ``` System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: The buffers supplied to a function was too small --- End of inner exception stack trace --- at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.AuthenticateAsClient(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation) at MailKit.Net.Imap.ImapClient.Connect(String host, Int32 port, SecureSocketOptions options, CancellationToken cancellationToken) at MailKit.MailService.<>c__DisplayClass3.b__2() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) at DownloadAndUpdatePrices.Email.EmailService.d__0.MoveNext() in d:\ProjectIS-3\WebProjectAutosouz\DownloadAndUpdatePrices\DownloadAndUpdatePrices\Email\EmailService.cs:line 35 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at DownloadAndUpdatePrices.BLL.DownloadService.d__0.MoveNext() in d:\ProjectIS-3\WebProjectAutosouz\DownloadAndUpdatePrices\DownloadAndUpdatePrices\BLL\DownloadService.cs:line 22 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at DownloadAndUpdatePrices.Program.d__d.MoveNext() in d:\ProjectIS-3\WebProjectAutosouz\DownloadAndUpdatePrices\DownloadAndUpdatePrices\Program.cs:line 48 ``` https://github.com/jstedfast/MailKit/issues/197 1855 area-System.Reflection dotNET 1861 area-Infrastructure Linux tests fail in CI due to mscorlib.ni.dll existing. Linux test runs in CI have been failing (silently) for a while with execute assembly errors. This started around here: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst/264/ Which had these changes: Update to latest CoreFx packages (commit: 08c0d8628ef621fbd44d7aaa85b3239616a6b108) Move DNU/Roslyn forward (commit: 804b4330a6ae767694cf89ebd173533b539f16f5) The problem looks to be due to the fact that we are now copying mscorlib.ni.dll into the test folder. This is not a Linux native image so things blow up when we try to load and run it. To unblock things, I am adding some logic to run-tests.sh to just delete the native images, but that's not the right long term thing (especially with crossgen support coming real soon now). We need to stop this incorrect native image from being deployed, and eventually deploy the right one. This issue tracks doing the right thing. Once fixed, we should remove the code in run-test.sh that's tagged with this issue number. 1863 area-System.Security Some OID tests failing Cross Platform These tests are failing in CI (which we previously were not catching because no tests were running) System.Security.Cryptography.Encoding.Tests.OidTests.TestFromFriendlyName [FAIL] System.Security.Cryptography.Encoding.Tests.OidTests.TestFromOidValue [FAIL] System.Security.Cryptography.Encoding.Tests.OidTests.TestKnownValues [FAIL] 1866 area-System.Net DNS protocol support enhancement: MX and SRV records Hi, Is there any plan to support MX or SRV DNS field in corefx ? Thank you. Bhaal22 1874 area-System.Diagnostics System.Diagnostics.ProcessTests.ProcessTest.Process_AsyncHalfCharacterAtATime test sometimes fails ``` System.Diagnostics.ProcessTests.ProcessTest.Process_AsyncHalfCharacterAtATime [FAIL] Assert.True() Failure Expected: True Actual: False Stack Trace: E:\oss\corefx\src\System.Diagnostics.Process\tests\System.Diagnostics.Process.Tests\Process_StreamTests.cs(131,0): at System.Diagnostics.ProcessTests.ProcessTest.Process_AsyncHalfCharacterAtATime() ``` 1877 area-System.Linq [System.Linq.Expressions] Allow Switch with empty cases list Now we can not create an expression, that represents this code: ``` cs switch(x) { default: return 0; } ``` But c# allows this and this is can be very helpful - users do not needs a checks and brunches in own code. Problem with a next line: https://github.com/dotnet/corefx/blob/master/src/System.Linq.Expressions/src/System/Linq/Expressions/SwitchExpression.cs#L205 I have asked this question on StackOverflow a few months ago: http://stackoverflow.com/questions/28360782/switch-without-cases-but-with-default-in-system-linq-expressions Why this is not allowed? Can we change it? 1880 area-System.IO Add event buffering to Windows FileSystemWatcher to avoid InternalBufferOverflowExceptions "Consumers should not have to worry about internals like the Windows FileSystemWatcher (FSW) trying to minimize usage of precious non-paged memory resulting in InternalBufferOverflowExceptions. By default the FSW should buffer events in ""normal"" memory. This could easily be implemented using a BlockingCollection. FSW could be configurable to use the old behaviour only. For details and and a implementation sample see my post: https://petermeinl.wordpress.com/2015/05/18/tamed-filesystemwatcher/ Average programmers (for which we should make life as easy as reasonable) often are often not even aware of having to implement an OnError event handler resulting in InternalBufferOverflowExceptions going unnoticed and FSW being perceived as unreliable. " 1883 area-System.Numerics Buffer too small in BigIntegerCalculator.Divide We are seeing some memory corruption after a recent change to biginteger.divide. This is what I got from an engineer investigating the issue: I was able to repro this bug and find its cause. The bug is in System.Runtime.Numerics.dll, introduced by change #1476305 on 5/23. There is an array overflow in unsafe C# code that does division of BigInteger, in System.Numerics.BigIntegerCalculator.Divide(UInt32_, Int32, UInt32_, Int32, UInt32_, Int32, UInt32_, Int32) in FxCore/Open/src/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.DivRem.cs. This method allocates a few arrays: ``` fixed (uint* dividend = new uint[dividendLength], divisor = new uint[divisorLength], guess = new uint[divisorLength]) ``` The guess array is used as the destination of a multiplication: MultiplyDivisor(divisor, divisorLength, digit, guess); This multiplies an array of length divisorLength with an uint. The result length is divisorLength+1, longer than the memory allocated for guess. This overwrites the sync block of the next item in memory, causing asserts later on. Increasing the allocation of guess by 1 should fix this bug. 1891 area-Meta What are the rules on adding new APIs to CoreFX? I saw this: https://github.com/dotnet/corefx/issues/1017#issuecomment-107752252. > To expose the RuntimeEnvironment class, it requires updation to the desktop api, which will not come to effect until next desktop release. Hence, the portable version cannot be released, before the desktop 4.6.1 is released, to ensure this contract works on all platforms, which is the same problem we had with PlatformID, we want to decouple ourselves from desktop dependencies and be more quick with releases. What are the rules that are proposed for updating .NET Core? There are at least a few scenarios: - Adding new members on a type that exists in the .NET Framework. - Adding new types that represent a similar concept that exists in the .NET Framework - Adding new types (new concepts) that do not exist in the .NET Framework. Ideally, we could update .NET Core at the pace we want. Is the issue that we might add one new member to a type that is represented in contract foo and doing so would make contract foo incompatible with .NET Framework vLatest? /cc @Priya91, @terrajobst, @davidfowl 1894 area-System.Net The message or signature supplied for verification has been altered "Hello. I'm using [MailKit](https://github.com/jstedfast/MailKit) And have simple code: ``` c# public static async Task DownloadFileAsync(EmailModel emailModel, bool notSetFlagSeen) { var result = new DownloadResult(); if (emailModel != null) { using (var client = new ImapClient()) { await client.ConnectAsync(emailModel.Host, emailModel.Port, emailModel.Secure); // Note: since we don't have an OAuth2 token, disable // the XOAUTH2 authentication mechanism. client.AuthenticationMechanisms.Remove(""XOAUTH""); await client.AuthenticateAsync(emailModel.UserName, emailModel.Password); ................................... ``` From time to time I receive this error message. Stack trace: ``` System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: The message or signature supplied for verification has been altered --- End of inner exception stack trace --- at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.AuthenticateAsClient(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation) at MailKit.Net.Imap.ImapClient.Connect(String host, Int32 port, SecureSocketOptions options, CancellationToken cancellationToken) at MailKit.MailService.<>c__DisplayClass3.b__2() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) at DownloadAndUpdatePrices.Email.EmailService.d__0.MoveNext() in d:\ProjectIS-3\WebProjectAutosouz\DownloadAndUpdatePrices\DownloadAndUpdatePrices\Email\EmailService.cs:line 35 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at DownloadAndUpdatePrices.BLL.DownloadService.d__0.MoveNext() in d:\ProjectIS-3\WebProjectAutosouz\DownloadAndUpdatePrices\DownloadAndUpdatePrices\BLL\DownloadService.cs:line 22 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) at DownloadAndUpdatePrices.Program.d__f.MoveNext() in d:\ProjectIS-3\WebProjectAutosouz\DownloadAndUpdatePrices\DownloadAndUpdatePrices\Program.cs:line 49 ``` " 1896 area-System.Diagnostics Process_MainModule failed in CI (returned different module than expected) http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_release_prtest/1532/console ``` System.Diagnostics.ProcessTests.ProcessTest.Process_MainModule [FAIL] Assert.Equal() Failure  (pos 0) Expected: corerun Actual: ntdll  (pos 0) Stack Trace: d:\j\workspace\dotnet_corefx_windows_release_prtest\src\System.Diagnostics.Process\tests\System.Diagnostics.Process.Tests\ProcessTest.cs(239,0): at System.Diagnostics.ProcessTests.ProcessTest.Process_MainModule() ``` 1899 area-System.Xml xmldocument load error "I am using nuget pacakge System.Xml.XmlDocument 4.0.0-beta-22819 and when I tried to load xml, I am getting the error `System.Xml.XmlException: Root element is missing.` for every valid xml file. Code ``` TextReader textReader = File.OpenText(@""xml\document.xml""); XmlReaderSettings settings = new XmlReaderSettings(); settings.IgnoreWhitespace = true; settings.DtdProcessing = DtdProcessing.Ignore; XmlDocument doc = new XmlDocument(); // doc.LoadXml(textReader.ReadToEnd()); using (StringReader sr = new StringReader(textReader.ReadToEnd())) using (XmlReader reader = XmlReader.Create(sr, settings)) { doc.Load(reader); } ``` XML files ``` University of XML Department of XML parsing George Washington Abraham Lincoln Bill J. Clinton George W. Bush Barack H. Obama Song Li ``` and another xml file ``` test test2 test3 ``` " 1900 area-System.Text Make sure object initialization in System.Text.Encodings.Web is synchronized See comments in https://github.com/dotnet/corefx/pull/1889 1903 area-System.Runtime "[Suggestion] Add ""Time-based"" Class to enhance the ""WeakReference""" "Hello all: In asp.net we have Cache, this can be automatically set ""timeout"" and release its memory by setting dependency such as ""Timespan"", ""file""..... However in other situations we don't have such functions yet. It would be better if we have such a ""Time-based"" class, and it will automatically release many ""short-period-timed"" variable, something like WeakReferece (However, WeakReference doesn't have a time-based foundation yet, and we cannot control how long it will be released). In my mind, this can be designed something like this following: ``` public sealed class TimeBasedValue :IDisposable where T:class { //An absolute time that bombs up the value public TimeBasedValue (T value, DateTime span){……} // Absolute/Relative time for bombing up the value public TimeBasedValue (T value, int timePeriod, Mode mode, Unit unit) {……} //When time is overdue, this value must be null public T Value{get;} //""Mode"" is an enum that shows how the time period is calculated: Absolute, Relative to the last access.... public Mode TimeMode{get;} // Hour, Minute, Second or Millsecond? public Unit Unit{get;} //This will be called once the time is overdue, and it will first check whether the Value has implemented ""IDisposable"" or not, if yes, call it first. However no matter what happens, the Value should be set to null public void Dispose() { var value = Value; IDisposable idisp = value as IDisposable ; try { if(idisp!=null) { idisp.Dispose(); } } finally { idisp = null; } } } ``` " 1904 area-System.IO zip_InvalidParametersAndStrangeFiles.StrangeFiles4 disabled on Unix "I'm disabling the System.IO.Compression.Tests.zip_InvalidParametersAndStrangeFiles.StrangeFiles4 test on Unix as it fails once we fix the Path.AltDirectorySeparatorChar to be '/' instead of '\' ('\' isn't a valid directory separator char on Unix). The problem stems from an assert in ZipTestHelpers: ``` C# Assert.Equal(file.Name, entry.Name); ``` This particular ""strange"" zip file (dataDescriptor.zip) contains an entry named ""notempty\second.txt"", and the assert is expecting the entry.Name to be ""second.txt"", but when '\' isn't a separator character, the entry.Name (which just returns Path.GetFileName(...)) is coming back as ""notempty\second.txt"". This is after all a ""strange"" zip file, so I don't know whether this is a product issue or a test issue. " 1914 area-System.Reflection Build more specific test verification methods for System.Reflection.Emit.ILGeneration constructor verification tests In CustomAttributeBuilderCtor1.PosTest1() ``` csharp CustomAttributeBuilder cab = new CustomAttributeBuilder( typeof(ObsoleteAttribute).GetConstructor(ctorParams), paramValues); Assert.NotNull(cab); ``` Instead of simpe Assert.NotNull, check for configuration of cab from the ctor. 1917 area-Serialization Fix XmlSerializer issue when deserializing type with protected parameterless constructor When deserialize types with non-public default constructor, generated code from XmlSerializer throws missing method exception because no constructor is available for Activator.CreateInstance. This is affecting debugging scenario for some apps. This is to port the change we did in NetNative. 1930 area-System.Net Regression in redirection functionality of System.Net.Http.HttpClient "@davidsh , I found this regression when trying to integrating `HttpClient` into `dnu restore` Here is a simple repro: ``` public class Program { public void Main(string[] args) { Task.Run(async () => { var client = new HttpClient(); //var request = new HttpRequestMessage(HttpMethod.Get, ""http://www.bing.com""); // Works var request = new HttpRequestMessage(HttpMethod.Get, ""http://bing.com""); // Throws exception using (var response = await client.SendAsync(request)) { Console.WriteLine(await response.Content.ReadAsStringAsync()); } }).Wait(); } } ``` The exception thrown: ``` System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The requested header was not found at System.Net.Http.WinHttpHandler.WinHttpStatusCallback(IntPtr handle, IntPtr context, UInt32 internetStatus, IntPtr statusInformation, UInt32 stat usInformationLength) --- End of inner exception stack trace --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() at ConsoleApp8.Program.<>c.<
b__0_0>d.MoveNext() in C:\AspNet\VS2015Projects\ConsoleApp8\src\ConsoleApp8\Program.cs:line 19 System.Net.Http.WinHttpException: The requested header was not found at System.Net.Http.WinHttpHandler.WinHttpStatusCallback(IntPtr handle, IntPtr context, UInt32 internetStatus, IntPtr statusInformation, UInt32 stat usInformationLength) ``` " 1931 area-Serialization Add support for data contract surrogate type aliasing 1936 area-System.Net System.System.Net.Http.FormUrlEncodedContent(...) can't handle very long parameters/values "FormUrlEncodedContent(IEnumerable>) [uses](https://github.com/dotnet/corefx/blob/5f3cf426df9d41e629a02ee6d8fb32089e35227b/src/System.Net.Http/src/System/Net/FormUrlEncodedContent.cs#L40) Uri.EscapeDataString which internally uses [UriHelper.EscapeString](https://github.com/dotnet/corefx/blob/f969471a5719e901e9147928c9a71ea17d6d7862/src/System.Private.Uri/src/System/UriHelper.cs#L128) which [can't handle](https://github.com/dotnet/corefx/blob/f969471a5719e901e9147928c9a71ea17d6d7862/src/System.Private.Uri/src/System/UriHelper.cs#L131) strings longer than 65520 characters. ``` C# var uc1 = new FormUrlEncodedContent( new Dictionary { {""test"", new string('F', 65520)} } );//works fine var uc = new FormUrlEncodedContent( new Dictionary { {""test"", new string('F', 65521)} } ); //exception ``` " 1942 area-System.Collections Proposal: Dictionary.TryAdd(TKey, TValue) It's very common to see code that does this: ``` C# Dictionary data = ...; ... if (!data.ContainsKey(key)) data.Add(key, value); ``` This forces the dictionary to lookup the key twice when the key isn't yet in the dictionary. It'd be great if we had an additional method on the type: ``` C# public bool TryAdd(TKey key, TValue value); ``` Functionally it would behave exactly as if it were implemented like: ``` C# public bool TryAdd(TKey key, TValue value) { if (ContainsKey(key)) return false; Add(key, value); return true; } ``` but the implementation internally would be done more efficiently, essentially implemented by calling a variant of Insert but that returns false instead of throwing when a duplicate key is found. 1961 area-System.Text Review all remaining samness tests in System.Text.Encodings.Web.Tests There are some tests in the suite that Assert.Same. These need to be reviewed and either removed or we need to ensure the code is synchronizing properly and the tests will not fail. 1963 area-System.Security Add EVP_CIPHER_CTX_new to System.Security.Cryptography.Native Interop.EVP.Cipher.cs: ``` // This was computed by sizeof(EVP_CIPHER_CTX) on a 64-bit Ubuntu // machine running OpenSSL 1.0.1f. If we end up making a native // interop boundary library for OpenSSL then a very good candidate // method would be EVP_CIPHER_CTX_new, so the size can be computed // to match the platform. internal const int EVP_CIPHER_CTX_SIZE = 168; ``` Currently the native library is in dotnet/coreclr; so some mild gymnastics might be required. EVP_CIPHER_CTX_new should allocate the memory and call (OpenSSL) EVP_CIPHER_CTX_init, and have a well-defined free. If gymnastics are required for checking this in, a followup issue to use the new method should be filed. If the native library has moved to dotnet/corefx by the time this is done, then the managed code to consume it should potentially be part of the same change. 1965 area-System.Security AES DecryptorReusabilty.TestDecryptorReusability test is failing on Unix ``` 14:20:43 Finished: System.Security.Cryptography.Encoding.Tests 14:20:43 System.Security.Cryptography.Encryption.Aes.Tests.DecryptorReusabilty.TestDecryptorReusability [FAIL] 14:20:43 Assert.Equal() Failure 14:20:43 Expected: Byte[] [20, 48, 113, 173, 237, ...] 14:20:43 Actual: Byte[] [20, 48, 113, 173, 237, ...] ``` 1967 area-System.Collections [Proposal] IImmutableSet.Union should handle duplicates like ISet.UnionWith The [`ISet.UnionWith`](https://msdn.microsoft.com/en-us/library/dd411713.aspx) method is documented to handle duplicates as follows: > ``` cs > void UnionWith(IEnumerable other) > ``` > > Any duplicate elements in _other_ are ignored. The [`Enumerable.Union`](https://msdn.microsoft.com/en-us/library/vstudio/bb341731.aspx) extension method documents an even stronger condition: > ``` cs > public static IEnumerable Union(this IEnumerable first, IEnumerable second) > ``` > > When the object returned by this method is enumerated, `Union` enumerates _first_ and _second_ in that order and yields each element that has not already been yielded. The `IImmutableSet.Union` method appears to provide the same functionality as the above method, but does not specify the behavior of the result with respect to duplicate elements. In the current implementation of `ImmutableSortedSet`, the `Union` operation behavior varies depending on the relative size of the `this` and `other` sequences. This could be confusing for users expecting the elements of the `this` sequence to take precedence. I propose defining the behavior with respect to duplicates partially (to match `ISet.UnionWith`) or fully (to match `Enumerable.Union`), and updating the implementation of `ImmutableSortedSet` to follow the new behavior. 1973 area-System.Collections Please add interface IReadOnlySet and make HashSet, SortedSet implement it "Since `IReadOnlyList` was added the parity between sets and lists has declined. It would be great to re-establish it. Using it would be an implementation-agnostic way of saying: ""here is this read-only collection where items are unique"". Clearly it's needed by many people: SQL Server: https://msdn.microsoft.com/en-us/library/gg503096.aspx Roslyn: https://github.com/dotnet/roslyn/blob/master/src/Compilers/Core/Portable/InternalUtilities/IReadOnlySet.cs Some Guy In Comments: http://blogs.msdn.com/b/bclteam/archive/2013/03/06/update-to-immutable-collections.aspx " 1975 area-System.Net Fixes issues with the System.Net.Http tests - Tests were not running because the directories were not named properly. - Rename test project to match the correct name. - Fix typo in resource strings. 1985 area-System.Security X509DistinguishedName::Format returns different answers on Windows and Unix The most concrete example is that OID(2.5.4.8) displays in Windows as 'S', but on Unix (OpenSSL) as 'ST'. This is related to https://www.github.com/dotnet/corefx/issues/1813, but a targeted fix for that issue would not fix this, and vice versa. 1991 area-System.IO Improve memory allocations of DeflateStream We use GZipStream to handle the data sent on the wire to a WebAPI with both big and small requests. The allocation stress is more evident with small packages though. On a sample of 60 seconds in an stress test we measured that System.IO.Compression.OutputWindow.ctor() and System.IO.Compression.DefrateStream.ctor() would be consistently in the top 5 allocators. Between the two 1.8Gb in allocations in 60 seconds. The root is that DefrateStream will create on every construction (1 per request) an Infrater (https://github.com/dotnet/corefx/blob/master/src/System.IO.Compression/src/System/IO/Compression/DeflateStream.cs#L62) and the Infrater will also build an OutputWindow (https://github.com/dotnet/corefx/blob/master/src/System.IO.Compression/src/System/IO/Compression/OutputWindow.cs). Which in turn will allocate 32Kb to use as a buffer for each request. Reusing those buffers and/or OutputWindow would aleviate a lot the allocation stress GZipStream causes when used in small streams contexts like Web APIs. 1993 area-System.Security [Feature] Finish X509Certificate support for Unix The current Unix X509 code was to the level that it needed to be to allow presenting an SSL wire cert for verification. This needs to be completed. The functions which currently throw a `NotImplementedException` are ``` Internal.Cryptography.Pal.CertificatePal::CertificatePal.FromFile(string, string, X509KeyStorageFlags) Internal.Cryptography.Pal.CertificatePal::CertificatePal.FromHandle(IntPtr) Internal.Cryptography.Pal.OpenSslX509CertificateReader::OpenSslX509CertificateReader.Archived.set(bool) Internal.Cryptography.Pal.OpenSslX509CertificateReader::OpenSslX509CertificateReader.FriendlyName.set(string) Internal.Cryptography.Pal.OpenSslX509CertificateReader::OpenSslX509CertificateReader.SetPrivateKey(AsymmetricAlgorithm, AsymmetricAlgorithm) Internal.Cryptography.Pal.OpenSslX509ChainProcessor::OpenSslX509ChainProcessor.ChainElements.get() Internal.Cryptography.Pal.OpenSslX509ChainProcessor::OpenSslX509ChainProcessor.ChainStatus.get() Internal.Cryptography.Pal.OpenSslX509ChainProcessor::OpenSslX509ChainProcessor.Verify(X509VerificationFlags, Exception) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.ComputeCapiSha1OfPublicKey(PublicKey) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.DecodePublicKey(Oid, byte[], byte[]) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.DecodeX509BasicConstraintsExtension(byte[], bool, bool, int) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.EncodeX509BasicConstraints2Extension(bool, bool, int) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.EncodeX509EnhancedKeyUsageExtension(OidCollection) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.EncodeX509KeyUsageExtension(X509KeyUsageFlags) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.EncodeX509SubjectKeyIdentifierExtension(byte[]) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.GetCertContentType(byte[]) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.GetCertContentType(string) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.X500DistinguishedNameEncode(string, X500DistinguishedNameFlags) Internal.Cryptography.Pal.OpenSslX509Encoder::OpenSslX509Encoder.X500DistinguishedNameFormat(byte[], bool) Internal.Cryptography.Pal.OpenSslX509StoreProvider::OpenSslX509StoreProvider.Add(ICertificatePal) Internal.Cryptography.Pal.OpenSslX509StoreProvider::OpenSslX509StoreProvider.Export(X509ContentType, string) Internal.Cryptography.Pal.OpenSslX509StoreProvider::OpenSslX509StoreProvider.Remove(ICertificatePal) Internal.Cryptography.Pal.StorePal::StorePal.FromBlob(byte[], string, X509KeyStorageFlags) Internal.Cryptography.Pal.StorePal::StorePal.FromCertificate(ICertificatePal) Internal.Cryptography.Pal.StorePal::StorePal.FromFile(string, string, X509KeyStorageFlags) Internal.Cryptography.Pal.StorePal::StorePal.FromSystemStore(string, StoreLocation, OpenFlags) Internal.Cryptography.Pal.StorePal::StorePal.LinkFromCertificateCollection(X509Certificate2Collection) ``` 1994 area-System.Net System.Net.Http package doesn't target 4.5.x TFMs in latest build "I'm using System.Net.Http-4.0.0-beta-23008 (afaik, the latest) and the TFMs in the package are ""DNXCore50"", ""netcore50"", and ""net46"". `lib/net46` directory in the latest package contains nothing but a `_._` empty file. Should there be a TFM for `dnx451` or similar? " 2003 area-Infrastructure Add option for building corefx in deterministic mode Roslyn will support compiling in [deterministic mode](https://github.com/dotnet/roslyn/issues/372) where same inputs generate same outputs. There should be an option of building corefx in this mode. Note: this depends on the deterministic build feature being implemented in Roslyn, there's nothing actionable for corefx for now, but I wanted to have something to track :) 2008 area-System.Security X509's Helpers::HexToByte needs to signal invalid input "Because HexToByte doesn't signal on invalid input DecodeHexString returns a byte[] for non-hex inputs. So when the FindByInvalidThumbprint test runs (and searches for ""Nothing"") that is the same as searching for `new byte[] { 0xFF, 0xFF, 0xFF, 0xFF }`. A more marginal case would be that ""00FZ00"" becomes `{ 0x00, 0xFF, 0x00 }`, which could cause some interesting wrong matches for things like FindBySerialNumber. As the Find API returns no matches (vs throwing exception) on invalid input, the Decode probably should be redone to TryDecode, giving the helper routines the knowledge that they should just return no match. " 2009 area-System.Security SyncRoot and IsSynchronized on X509 collections should be explicitly implemented (private) I noticed that [`X509ChainElementCollection`](https://github.com/dotnet/corefx/blob/master/src/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509ChainElementCollection.cs#L24-L32) and [`X509ExtensionCollection`](https://github.com/dotnet/corefx/blob/master/src/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509ExtensionCollection.cs#L28-L36) implement `ICollection`'s `SyncRoot` and `IsSynchronized` properties implicitly (public). These should be explicit (private). I believe this is a compatible change from a subsetting perspective (between desktop and corefx) as this was done on the Regex collections, for example. Other collections in this assembly, like [`X509CertificateCollection`](https://github.com/dotnet/corefx/blob/master/src/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509CertificateCollection.cs#L43-L54) already implement these properties explicitly (private). I'd be happy to submit a PR. cc @bartonjs 2011 area-System.IO Several FileSystemWatcher tests are failing on OSX Expected events are not being raised for at least: - FileSystemWatcher_Changed_LastWrite_File - FileSystemWatcher_Changed_LastWrite_Directory - FileSystemWatcher_Created_MoveDirectory A Debug.Assert in FindRenameChangePairedChange is also firing for the MoveDirectory test: 2022 area-System.IO What about adding interfaces to FileInfo and DirectoryInfo? When testing it would be nice to be able to mock the filesystem. Would a PR introducing interfaces be accepted? 2028 area-System.Collections Inconsistent, unexpected bounds check in RemoveRange - `ImmutableArray.RemoveRange` requires 0<=index.RemoveRange` requires 0<=index.Builder.ToImmutable Currently the documentation for `ImmutableList.Builder.ToImmutable()` includes the following remark: > This method is an O(n) operation, and approaches O(1) time as the number of actual mutations to the set since the last call to this method approaches 0. The problem with this statement is the complexity of this operation is only O(1) when the number of mutations is _exactly_ equal to 0. As the number of mutations _approaches_ 0, the complexity is actually approaching O(log n). 2036 area-System.Collections Exposed Immutable Collections APIs should allow `null` comparers Several immutable collections APIs throw an `ArgumentNullException` if a null `IComparer` or `IEqualityComparer` is provided. This is not intuitive, and deviates from a very common pattern established by the .NET framework. All publicly-exposed API methods in this assembly should be updated to ensure the following: 1. If a null value is provided for `IComparer`, `Comparer.Default` should be used instead. 2. If a null value is provided for `IEqualityComparer`, `EqualityComparer.Default` should be used instead. 2046 area-System.IO How to implement System.IO.FileSystem.Watcher on FreeBSD This issue is to discuss how we can implement `System.IO.FileSystem.Watcher` on FreeBSD which does not have inotify: - [ ] Determine/discuss if this be a source of trouble? - [ ] We could stub it out at the moment and throw a PlatformNotsupportedException as suggested in by Stephen - https://github.com/dotnet/corefx/pull/2021#issuecomment-111602342 - [ ] Implement proper solution using kqueue/libev? Any recommendations here are very much welcome. # Reference Material - https://www.freebsd.org/cgi/man.cgi?query=kqueue - http://software.schmorp.de/pkg/libev.html - http://libevent.org/ # Related Issues - https://github.com/dotnet/corefx/issues/1626#issuecomment-111975933 - https://github.com/dotnet/corefx/issues/1626#issuecomment-111975140 2051 area-System.Reflection [PortablePdb] Local signature in method body, entry point, #Pdb stream Adds local signature token to method body, so that PDB readers have access to them without loading IL stream. Adds a new #Pdb stream to standalone debug metadata that includes sizes of type-system tables referenced by the debug metadata tables. Replaces entry point CDI with an entry in #Pdb stream. 2053 area-Serialization Fix exception message differences when deserializing empty string There is differences in the exception message when deserializing empty string using DataContractSerializer in NetCore/NetNative compared to Desktop. This is due to XmlDictionaryReader uses different kind of XmlReader in one of the CreateTextReader overloads. This PR updates the overload to initialize to the same XmlUtf8TextReader. 2058 area-System.Reflection Merge branch 'master' of https://github.com/dotnet/corefx into dev/metadata 2061 area-System.Reflection [MetadataReader] document format and constrains of -Ptr tables InitializeTableReaders method contains the following code: ``` C# // Compute ref sizes for tables that can have pointer tables int fieldRefSizeSorted = GetReferenceSize(rowCounts, TableIndex.FieldPtr) > SmallIndexSize ? LargeIndexSize : GetReferenceSize(rowCounts, TableIndex.Field); int methodRefSizeSorted = GetReferenceSize(rowCounts, TableIndex.MethodPtr) > SmallIndexSize ? LargeIndexSize : GetReferenceSize(rowCounts, TableIndex.MethodDef); int paramRefSizeSorted = GetReferenceSize(rowCounts, TableIndex.ParamPtr) > SmallIndexSize ? LargeIndexSize : GetReferenceSize(rowCounts, TableIndex.Param); int eventRefSizeSorted = GetReferenceSize(rowCounts, TableIndex.EventPtr) > SmallIndexSize ? LargeIndexSize : GetReferenceSize(rowCounts, TableIndex.Event); int propertyRefSizeSorted = GetReferenceSize(rowCounts, TableIndex.PropertyPtr) > SmallIndexSize ? LargeIndexSize : GetReferenceSize(rowCounts, TableIndex.Property); ``` It's not clear if any of these conditions are ever true. I'd think that the size of XxxPtr table is always the same as the corresponding Xxx table, since the Ptr table is used as an index. It might make sense that XxxPtr table is smaller than the Xxx table. But I don't see how it can have more rows. Do we have tests that cover these? 2064 area-System.Runtime INumber or INumeric interface for base number types I often run into the problem that I want to handle numeric base types in a common way. Unfortunately, they are missing a common interface to express their similarities. Therefore I propose to create an `INumber` or `INumeric` interface on the following numeric base types: - `System.Decimal` - `System.Double` - `System.Single` - `System.Int32` - `System.UInt32` - `System.Int64` - `System.UInt64` This interface should define common functionality, like `+`, `-`, `*`, `/`, etc. 2065 area-System.Data Are DataColumn, DataTable and DescriptionAttribute supported with ASP .Net Core ? I am trying to compile my existing app with ASP .Net Core 5.0. During compilation several error we are getting error like : error CS0246: The type or namespace name 'DescriptionAttribute' could not be found (are you missing a using directive or an assembly reference?) etc. Aren't these supported ? 2069 area-System.Reflection Disable tests failing in internal build Tracked by #1787 2070 area-System.Diagnostics How to implement System.Diagnostics.Process on FreeBSD? This issue is to discuss how we can implement `System.Diagnostics.Process` on FreeBSD which does not have procfs by enabled by default as [it should be considered deprecated.](http://lists.freebsd.org/pipermail/freebsd-fs/2011-February/010760.html) # Reference Material - [Implement basic System.Diagnostics.Process.Modules support on Linux #2188](https://github.com/dotnet/corefx/pull/2188#issuecomment-116671221) - The System.Diagnostics.Process [specification on MSDN.](https://msdn.microsoft.com/en-us/library/system.diagnostics.process%28v=vs.110%29.aspx) - The [FreeBSD process information implementation within Mono](https://github.com/mono/mono/blob/master/mono/io-layer/processes.c#L1888-L1910) (LGPL) - The [FreeBSD process information implementation within IO.JS](https://github.com/nodejs/io.js/blob/master/deps/uv/src/unix/freebsd.c) (MIT) - https://www.freebsd.org/cgi/man.cgi?procstat and https://www.freebsd.org/cgi/man.cgi?sysctl(3) ``` KERN_PROC Return selected information about specific running processes. For the following names, an array of struct kinfo_proc structures is returned, whose size depends on the current number of such objects in the system. Third level name Fourth level is: KERN_PROC_ALL None KERN_PROC_PID A process ID KERN_PROC_PGRP A process group KERN_PROC_TTY A tty device KERN_PROC_UID A user ID KERN_PROC_RUID A real user ID If the third level name is KERN_PROC_ARGS then the command line argument array is returned in a flattened form, i.e., zero-termi- nated arguments follow each other. The total size of array is returned. It is also possible for a process to set its own process title this way. If the third level name is KERN_PROC_PATHNAME, the path of the process' text file is stored. For KERN_PROC_PATHNAME, a process ID of -1 implies the current process. Third level name Fourth level is: KERN_PROC_ARGS A process ID KERN_PROC_PATHNAME A process ID ``` /cc: @janhenke @saper @jasonwilliams200OK @josteink 2075 area-System.Linq Append/Prepend "## Summary Two of the IEnumerable extension methods that I carry in my collection (we all have a personal collection of IEnumerable extension methods, am I right?) are Append and Prepend, for adding a single element to the end or beginning of a sequence respectively. Though not quite as common as concatenating sequences, I have been surprised at how often this need arises. Doing this inside of an expression today is not straightforward; the best equivalent that I have found is far from ideal: ``` sequence.Concat(Enumerable.Repeat(singleValue, 1)) ``` I propose adding Prepend and Append extension methods to Enumerable to fill this need. ## API ``` public static class Enumerable { public static IEnumerable Append(this IEnumerable source, T value); public static IEnumerable Prepend(this IEnumerable source, T value) } ``` ## Usage ``` optionalValues.Prepend("""") ``` " 2076 area-System.Linq Sorting sequences ## Summary There are two scenarios where the existing Enumerable OrderBy methods are not ideal: - When sorting a sequence of comparable values (e.g. ints). In this case an identify function must be provided as the key selector: `ints.Sort(x => x)` - When sorting on something other than a single key, an identity function must be provided to select the element as the key, AND an instance of IComparer must be provided, which is at odds with the majority of LINQ methods which take functions (and are therefore easier to use with lambda expressions): ``` using NodaTime; using NodaTime.TimeZones; ... IEnumerable tzs = ...; var tzEqComparer = new ZoneEqualityComparer(); tzs = tzs.OrderBy(x => x, Comparer.Create((tz1, tz2) => tzEqComparer.Equals(tz1, tz2) ? 0 : tz1.Id.CompareTo(tz2.Id)); ``` I propose adding various overloads of Sort to address these situations. The name Sort is chosen specifically to distinguish these methods, which do not take a key selector, from the OrderBy methods, which do. If this is not considered an important distinction, the names could be changed to OrderBy without colliding with the existing methods. ## API ``` public static class Enumerable { public static IOrderedEnumerable Sort(this IEnumerable source) where T : IComparable public static IOrderedEnumerable Sort(this IEnumerable source, IComparer comparer) public static IOrderedEnumerable Sort(this IEnumerable source, Comparison comparison) public static IOrderedEnumerable SortDescending(this IEnumerable source) where T : IComparable public static IOrderedEnumerable SortDescending(this IEnumerable source, IComparer comparer) public static IOrderedEnumerable SortDescending(this IEnumerable source, Comparison comparison) public static IOrderedEnumerable ThenSort(this IOrderedEnumerable source) where T : IComparable public static IOrderedEnumerable ThenSort(this IOrderedEnumerable source, IComparer comparer) public static IOrderedEnumerable ThenSort(this IOrderedEnumerable source, Comparison comparison) public static IOrderedEnumerable ThenSortDescending(this IOrderedEnumerable source) where T : IComparable public static IOrderedEnumerable ThenSortDescending(this IOrderedEnumerable source, IComparer comparer) public static IOrderedEnumerable ThenSortDescending(this IOrderedEnumerable source, Comparison comparison) } ``` ## Usage ``` ints.Sort() ``` ``` using NodaTime; using NodaTime.TimeZones; ... IEnumerable tzs = ...; var tzEqComparer = new ZoneEqualityComparer(); tzs = tzs.Sort((tz1, tz2) => tzEqComparer.Equals(tz1, tz2) ? 0 : tz1.Id.CompareTo(tz2.Id)); ``` ## Questions 1. The generic restriction to IComparable is not necessary if the implementation will delegate to OrderBy with an identity function (although there are other potential implementations). However, even if not strictly necessary, it clearly expresses the intent of the method. Is there a downside to keeping it? 2. Overloads that take an IComparer are included for parity with OrderBy. Between the two, I prefer the overload that takes the Comparison, since converting an IComparer into a Comparison is more straightforward than going the other way. However, I am not aware of the rationale behind using IComparer instead of Comparison in the OrderBy methods. We should consider whether the same rationale applies to Sort, or whether it might make sense to remove the IComparer overloads to reduce the API surface. 3. Should the OrderBy methods have overloads added that take Comparison for parity? Or is there something about the usage pattern of selecting a key that makes it less likely that a comparison function will be provided inline? (This is the inverse of question 2). 4. What about Queryable? 2077 area-Serialization Support JSON-LD in DataContractJsonSerializer There is an issue raised by dotnet/wcf#28 requesting support for JSON-LD. If more info is needed, @akuckartz would be a good person to start with, as the original issue-raiser. It would be great to hear from the community if JSON-LD is useful so this work can be prioritized appropriately. Thanks! 2081 area-Serialization Update NetCore DCJS to use the same code with NetNative DCJS DCJS implementation in NetCore at the moment is based on the JavascriptSerializer/Deserializer which is rather different from the ones used in NetNative and on Desktop. This introduces unneccessary effort in making improvement and providing consistent behaviors. This change switches DCJS in NetCore to the same code that is used in NetNative with MERGE_DCJS directive. If there is no issues I will follow to clean up this directive and unused code. @SGuyGe @zhenlan @roncain @shmao 2085 area-Serialization Fix serialization of generic queue to align with NetNative Generic queue is serialized differently in NetCore compared to NetNative since it skips the queue's _syncRoot property of type object. This change is to fix the behavior when serializing property of type object in general. As Serialization of generic queue on desktop is controlled with Serializable, NonSerialized attributes, which are not available in NetCore, this results in a different load compared to desktop. I have verified that cross platform serialization/deserialization works fine. cc: @SGuyGe @zhenlan @shmao 2093 area-System.ComponentModel Can `sealed` modifier be removed from `DisplayAttribute`? I'm redirected here from MVC6 repo (https://github.com/aspnet/Mvc/issues/2706). In MVC6 we are meant to use `DisplayAttribute` to define human readable texts for model's properties. In earlier versions, MVC was using `DisplayNameAttribute` and it could be inherited, so developers were overriding it and customizing texts, messages, labels on the air (like translating or formatting them etc.). Now MVC6 uses .NET's `DisplayAttribute` for this use and it's sealed and unfortunately we are restricted here now. Is it possible to remove the `sealed` modifier from this attribute? So I think the frameworks using this attribute will be more customizable. 2101 area-System.Reflection Add TypeInfo.IsEquivalent(TypeInfo) to CoreFx It is currently not possible to write a library that compares types correctly (considering NoPIA interface unification) and that works on both Desktop CLR and CoreCLR without calling Type.IsEquivalentTo via Reflection on Desktop. 2102 area-System.Text Consider adding StringBuilder.AppendEncoded(Encoding encoding, byte* bytes, int byteCount) In general Encoding APIs don't compose well with other fundamental APIs without allocating temporary blobs/strings in between. 2103 area-System.Globalization How should we handle default data for Globalization tests? "In .NET Core, a large portion of the implementation of our globalization code is built on-top of OS primitives. which we don't control. Globalization data changes frequently and updates to the data can break unit tests. Some examples of changes we've had problems with in the past: 1. Collation data changing across releases and platforms, causing sorting tests to break. 2. Locale data changing (e.g a locale adopts a new currency symbol) causing test breaks. 3. Casing data changes, breaking ToUpper/ToLower tests. 4. Different platforms supporting different version of IDNA. In our internal tests, we've taken a few approaches to solving these problems: 1. Implementing another set of (hopefully lightweight) P/Invokes on-top of OS Functions and using them to generate the expected data at runtime. 2. Disabling specific tests (by hand) on specific platforms or changing the expected data for a specific platform via a runtime test. 3. Encoding metadata about the version of Unicode a platform supports and the version a test requires and skipping tests if they don't match. As I start to open source more of our collation tests, I'm running into problems where we depend on support libraries that implement the OS behavior from (1). It's not clear to me that adopting this is the right long term approach. There's a bunch of platform specific code in these support libraries which will also have to be brought cross platform. The library also contains a fair amount of logic itself. My general preference would be to do something like the following: 1. Use XUnit data driven tests (i.e. `[Theory]`) to write tests that decouple the implementation of the test from the input data. 2. Using `[MemberData]` control the set of inputs that flow to each test based on runtime platform detection. For example, consider something like the following: ``` csharp [Theory] [MemberData(""CompareToData"")] public static void CompareTo(string localeName, string left, string right, int expected, CompareOptions options) { CompareInfo ci = CompareInfo.GetCompareInfo(localeName); int result = ci.Compare(left, right, options); Assert.Equal(expected, Math.Sign(result)); } public static IEnumerable CompareToData { get { yield return new object[] { """", ""Works Everywhere"", ""WORKS EVERYWHERE"", 0, CompareOptions.IgnoreCase }; if (PlatformDetection.IsWindows) { yield return /* ... some set of data that only works on Windows ... */; } } } ``` Will will also need the ability to run certain cases on a specific version of Windows (since behavior changes between versions) or a specific version of ICU. However, I would expect the general shape of the tests would remain the same (`[Theory]`'s + `[MemberData]`'s) and the only thing that would differ would be the filtering we use when deciding if we include a test case in the MemberData. I think there are two downsides with this approach: 1. Bringing a new version of an existing platform online is more expensive. Since we no longer call to the OS to get our oracle data, we will need to investigate test failures to understand if there are product bugs or simply changes in OS algorithms. 2. `[Theory]` is slower than simply having a ton of a `Assert`'s for different cases (or doing a loop) in one `[Fact]`. The major slowdown was in test discovery. As an experiment (and to ease some debugging for some work I was doing), I ported the [NormalizationAll](https://github.com/dotnet/corefx/blob/master/src/System.Globalization.Extensions/tests/Normalization/NormalizationAll.cs) test to use `[Theory]` and the test which used complete very quickly now take about a minute to run (There are 72K test cases, XUnit reports that it took 10.6 seconds to execute the tests (compared to 1.5 when using a loop), but that didn't count the ~50 seconds to discover the tests). It also appears that much of this time is spent even if the tests would not be run (e.g due to `[ActiveIssue]` or `[OuterLoop]`). Ideally this could be fixed, but I am not sure, naively it feels weird that you pay an order of magnitude penalty in test execution because of Theory, but I have not dug into the internals of xUnit. The mitigation here is to not use [Theory] when doing so would generate tons of cases, I guess, and just do a loop ourselves (ideally still over something like a member data API). I'm not sure if these are dealbreakers or not. (2) seems bad, but is a general problem for us adopting Theory for large data-sets, which seems to be a a problem for more than just globalization. I also really hope that we can adopt something like this, since it makes reporting for data driven tests so much nicer. I'm interested in what @stephentoub and @tarekgh think here. Does the pain of OS Data updates make the OS Wrapper library the right approach, even if it brings in a lot of extra code and causes a another large amount of work when we bring up a new platform? Is the fixed data + filtering approach a better long term direction? Do we need both for some reason? " 2106 area-System.Security Proposal: Add Ed25519 and Curve25519 Algorithms to System.Security.Cryptography The System.Security.Cryptography namespace contains several commonly used cryptographic algorithms. I propose the addition of two commonly used algorithms which are currently missing: - **Ed25519** is an implementation of an _Edwards-curve Digital Signature Algorithm_. - **Curve25519** is an algorithm designed to be used with _elliptic curve Diffie-Hellman (ECDH) key exchange algorithm_. Both are based on the same elliptic curve. Many open source projects (e.g. OpenSSH, OpenBSD) seem to prefer this elliptic curve for use in elliptic curve cryptography (ECC). It is therefore beneficial to have these algorithms implemented in CoreFX. The two algorithms should be included in the existing classes in the `System.Security.Cryptography` namespace, but it must be possible to explicitly ask for the usage of these algorithms. 2108 area-Meta Where should ref docs and associated samples be stored? Ideally, they would live in the code repo. Thoughts? For clarity, ref docs augment the metadata and /// comments already available in the repo/builds. Many topics won't have ref topics. @blackdwarf, @danroth27 In reference to #2107 2109 area-System.Reflection Add System.Reflection.Metadata.Test.Desktop, which targets FX 4.6 and Xunit 1.9.2 This makes it possible to run and debug tests in VS. Caveats: can't use Theory attribute since it's not available in Xunit 1.9.2 2110 area-System.Reflection TD.NET doesn't work well with shared projects, replacing with manual sharing 2114 area-System.Linq Enumerable.SequenceEqual takes IEqualityComparer rather than a predicate Why does [Enumerable.SequenceEqual](http://msdn.microsoft.com/en-us/library/vstudio/bb342073.aspx) take its comparer as an [IEqualityComparer](http://msdn.microsoft.com/en-us/library/vstudio/ms132151.aspx)? The algorithm seems to make no use of [GetHashCode](http://msdn.microsoft.com/en-us/library/vstudio/ms132155.aspx). Why does it not instead take a `Func` predicate, similar to how [First](http://msdn.microsoft.com/en-us/library/vstudio/bb535050.aspx) takes a `Func`? 2119 area-System.Runtime Make all structs that claim to be thread-safe dereference `this` at most once per entrypoint `ImmutableArray` goes to great lengths to be thread-safe by [never dereferencing `this` more than once per public entrypoint](https://github.com/dotnet/corefx/blob/1dc118cc46f88e4889a23190b450d89e565967fd/src/System.Collections.Immutable/src/System/Collections/Immutable/ImmutableArray%601.cs#L18-L33) (to fix bug #191). Other structs in the Framework do _not_ take this care (such as `DateTime`, `Nullable`, or `System.Int32`), despite [claiming to be thread-safe in MSDN documentation](https://msdn.microsoft.com/en-us/library/System.DateTime%28v=vs.110%29.aspx). Should we take the same care for the other structs? The community [has been fascinated](http://stackoverflow.com/questions/30957211/what-advantage-is-there-to-storing-this-in-a-local-variable-in-a-struct-method/30957262) by the pattern in `ImmutableArray` and brought up that it might make sense to spread it around more. 2121 area-System.Reflection Implements fast document name comparison, parsing, and general case-insensitive string comparison 2124 area-Infrastructure We need a way to run tests on different runtime We need a way to choose between different platforms when running tests. Example proposition: ``` build /p:TestProduct=Desktop ``` 2127 area-System.Collections Consider adding KeyValuePair type with a factory method For better or worse KeyValuePair is frequently used structure to represent a pair (mostly due to the lack of struct tuples). It is currently inconvenient to create these pairs as one needs to specify type arguments to the constructor. A simple factory method would make it more succinct since the C# and VB compilers can usually infer the generic arguments: ``` C# public static class KeyValuePair { public static KeyValuePair Create(K key, V value) { return new KeyValuePair(key, value); } } ``` this pattern is already used by ImmutableArray. 2129 area-Serialization Fix Jit assert failure in struct serialization tests due to missing cast This change fixes 3 struct serialization test Jit assert failures due to missing cast in IL generated code. cc: @zhenlan @SGuyGe @shmao 2135 area-Infrastructure Reenable build multithreading once dnx issues are fixed Once dnx concurrency issues are fixed, reenable project multithreading. Currently disabled in CI using /p:SerializeProjects=false. 2139 area-System.Net Merge changes from TFS 2146 area-System.Linq Add commonly required Enumerable methods (DistinctBy, ExceptBy, AsChunked, ...) We have Distinct, Except, Intersect, Union, etc. Often it is required, though, to base the set operation on some key. OrderBy can do that for example. I request that the following methods be added: - DistinctBy (pick the first element deterministically for each key) - ExceptBy (return all items of the first sequence that do not have a matching key in the second) - IntersectBy (return the first item in a.Concat(b) for each unique key where the key appears in both collections) - UnionBy (return the first item in a.Concat(b) for each unique key) - MaxBy/MinBy The set operations need to deal with duplicate keys in a sensible way. I have added to that list my idea for how to do that in sane and well-defined way.
Another commonly required helper method is one that splits a sequence into chunks of a fixed size: ``` static IEnumerable> AsChunked(this IEnumerable sequence, int chunkSize) ``` This is useful for many things, for example: - Split a list into pages - Make AsParallel operate on coarser items by feeding it chunks - If you want to save a huge stream of items to files you can use this to split the sequence into one chunk per file - Request rows from the database in chunks of 1000 IDs This requirement comes up on Stack Overflow a lot and there is no built-in way to do it. The chunks should be drawn lazily. In fact the items of each chunk should be lazy as well. This allows for very big chunks such as 1 billion items per chunk. Maybe we need two variants: 1. `IEnumerable> AsChunkedLazy` 2. `IEnumerable AsChunkedEager` The second one would be easier to handle for consumers. The first one requires more disciplined use because each chunk must be enumerated at most once and the chunks must be enumerated in order. This is required for an efficient streaming operation. 2149 area-System.Net Initial open source WinHttpHandler This is the initial open sourcing of the System.Net.Http.WinHttpHandler package. This commit also includes a fully buildable System.Net.Http.dll binary as well as System.Net.Http.WinHttpHandler.dll binary. Unit tests are included as well. There is a skeleton file for beginning cross-platform work on System.Net.Http. 2150 area-Infrastructure Unit tests on Windows run on 32-bit CoreCLR We are using a 32bit coreclr to run all of our tests when running on windows. This is different from what we do for OSX and Linux (where we use the 64 bit version of the runtime built from [dotnet/coreclr](https://github.com/dotnet/coreclr)). I believe because of this we also don't get coverage on any of the RyuJIT specific optimizations around Vectors as part of our unit tests. I think it would be ideal if we also ran tests on a 64 bit runtime and it corresponded to some LKG build from [dotnet/coreclr](https://github.com/dotnet/coreclr). /cc @stephentoub 2151 area-System.Reflection Fix incorrect MethodSpec signature header check There was some confusion in untested code around ELEMENTTYPE_GENERICINST vs. CALLCONV_GENERICINST cc @tmat 2152 area-System.Xml [Perf] Fix XmlDocument/XPath to properly compare strings by ref "XmlDocument/XPath when comparing namespaces it uses fixed string object (from private class defined in XmlConst.cs). During the split of System.Xml.dll to smaller libraries (XmlDocument, ReaderWriter, XPath, XPath.XmlDocument) we introduced shared XmlConst.cs file for those constants and thus different references for those strings. To optimize implementation is comparing built-in namespaces by reference and thus in some cases it doesn't work properly and causes some undefined behavior. The cases which were breaking tests were already fixed although there are some other broken scenarios which got introduced when running product on Desktop (runtime). Steps: - [x] Workaround to not compare by ref (one line change in Ref.cs) to unblock Desktop ASAP (we are gonna lose some perf but correctness is more important than perf) - https://github.com/dotnet/corefx/pull/2154 - [ ] Make sure there is only one copy of string constants in one of the libraries (ReaderWriter or XmlDocument) and other libraries should get the valid and needed references to string some other way (i.e. through XmlDoc's NameTable or exposing something new in the contract if NameTable is too slow or not possible) - [ ] Re-enable comparison by ref and validate it is passing when running on Desktop runtime (on Win 10) and coreclr **Except for the first checkbox it is up for grabs** Please revert Ref.cs to its original form with the last checkbox: ``` csharp public static bool Equal(string strA, string strB) { #if DEBUG if (((object)strA != (object)strB) && string.Equals(strA, strB)) System.Diagnostics.Debug.Fail(""Ref.Equal: Object comparison used for non-atomized string '"" + strA + ""'""); #endif return (object)strA == (object)strB; } ``` " 2153 area-System.Net Eliminate closure allocations in System.Net.Http 2155 area-System.Net System.Net.Http.dll should throw NotImplementedException instead of DllNotFoundException When sending HTTP request with HttpClient on CoreCLR + *nix, the following exception is thrown: ``` Unhandled Exception: System.DllNotFoundException: Unable to load DLL 'winhttp.dll': Error 126 (Exception from HRESULT: 0x8007007E) at System.Net.Http.Interop.WinHttpSetStatusCallback(SafeInternetHandle handle, WINHTTP_STATUS_CALLBACK callback, UInt32 notificationFlags, IntPtr reserved) at System.Net.Http.WinHttpHandler.SetStatusCallback(SafeInternetHandle requestHandle, WINHTTP_STATUS_CALLBACK callback) at System.Net.Http.WinHttpHandler.d__1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.AsyncMethodBuilderCore.b__4(Object state) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() at System.Threading.ThreadPoolWorkQueue.Dispatch() libc++abi.dylib: terminating with uncaught exception of type PAL_SEHException Abort trap: 6 ``` It is better to use a more friendly exception. 2159 area-System.Linq Overridden ToString not being called (IOrderedQueryable) I have a class which implements `IOrderedQueryable`. With DNX 4, I can print out my LINQ queries as described in [this LINQ provider tutorial](http://blogs.msdn.com/b/mattwar/archive/2007/07/31/linq-building-an-iqueryable-provider-part-ii.aspx) (i.e. `Console.WriteLine(ExampleQueryable.Where(x => ...))` should call the overridden `ToString()` on my class implementing `IOrderedQueryable`). It works fine in DNX 4, but with DNX Core, it skips my overridden `ToString()` completely (I tested this by placing a breakpoint), and instead seems to be calling `ToString()` on `System.Object`. 2165 area-System.Net Follow-up feedback/action items for WinHttpHandler This issue is capturing feedback from the PR which copied the sources of WinHttpHandler into GitHub. https://github.com/dotnet/corefx/pull/2149 While most of the original feedback was addressed in the PR, some feedback action items are deferred and are being tracked in this issue. - [x] Review use of SafeWinHttpHandle. Check for leaks etc. - Tracking with #2501 - [x] Review TODO's in code. - Done. All TODO's have separate issues. - [x] Consolidate use of Safe*Handle code especially SafeHandleZeroOrMinusOneIsInvalid classes - Tracking with #2501 - [x] Optimize error handling around chain.Build(). - Done #4910 - [x] Optimize closures in Task related lambdas - Done #3867 - [x] Optimize use of TaskCompletionSource regarding `RequestState` - Done #3867 - [x] Propagate `CancellationToken` in calls to `TrySetCanceled` Fixed w/ #2608 - [x] Pass `CancellationToken` to `CopyToAsync` - Considered but `HttpContent.CopyToAsync` does not have an overload that takes a `CancellationToken`. - [x] Consider moving `HRESULT_FROM_WIN32` method to Common/Interop folder. - Done #2511 - [x] Consider change DLL import signature of `GetModuleHandle` to return `SafeHandle` or use `GetModuleEx` with `GET_MODULE_HANDLE_EX_FLAG_PIN` or `GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT` - Considered but there are no race conditions since the 'winhttp.dll' is always loaded during the use of the WinHttpHandler class. - [x] Add UnitTests to test fix with proxy authentication and redirection for PR #2463 - [x] Move duplicate code in WinHttpCertificateHelper.cs and merge with similar code used in System.Net.Security - Tracked in #14542. - [x] Optimize auth scheme lookups in WinHttpAuthHelper.cs - WON'T FIX. - [x] Register callback on cancellation token to cancel WinHTTP operation for WinHttpRequestStream, WinHttpResponseStream - Issue #8627 for WinHttpResponseStream - [x] Add logging for failures in classes such as WinHttpResponseParser, WinHttpResponseStream - Tracked in #7856. - [x] Consider refactoring cache logic from WinHttpAuthHelper into separate class and avoid out parameters - WON'T FIX. - [x] Optimize algorithm used in scanning client certificate list in WinHttpCertificateHelper - WON'T FIX. - [x] Refactor cookie handling from WinHttpHandler into separate class. - Done #5001 2167 area-System.Collections Add ImmutableArray.Builder.Sort(Comparison comparison) List has an overload of Sort that takes a delegate. It's useful when writing an inline comparer using a lambda: ``` C# list.Sort((x, y) => ); ``` 2168 area-System.Net Sporadic wrong response from HttpClient.SendAsync() when using an authenticated proxy "@davidsh Repro steps: 1. Fiddler Web Debugger -> Rules -> Require Proxy Authentication 2. Run the following program and at some point you will see: ![capture](https://cloud.githubusercontent.com/assets/1383883/8366945/bdfdbc32-1b4e-11e5-9a4a-3c336f95fe52.PNG) Looks like you get `407 (Proxy Auth Required)` sporadically even you always provide correct credentials for the proxy. Can only repro with CoreCLR. ``` c# using System; using System.Net; using System.Net.Http; using System.Threading.Tasks; using System.Collections.Generic; namespace ConsoleApp3 { public class Program { public void Main(string[] args) { var webProxy = new WebProxy(""http://localhost:8888""); webProxy.Credentials = new NetworkCredential(""1"", ""1""); var handler = new HttpClientHandler { Proxy = webProxy, UseProxy = true }; var client = new HttpClient(handler); var tasks = new List(); foreach (var uri in URIs.Split(new[] { Environment.NewLine }, StringSplitOptions.RemoveEmptyEntries)) { tasks.Add(SendRequest(client, uri)); } Task.WhenAll(tasks).Wait(); Console.WriteLine(""Done""); } private async Task SendRequest(HttpClient client, string uri) { Console.WriteLine($""GET {uri}""); var request = new HttpRequestMessage(HttpMethod.Get, uri); using (var response = await client.SendAsync(request)) { Console.WriteLine($""{response.StatusCode} response from {uri}""); response.EnsureSuccessStatusCode(); } } private class WebProxy : IWebProxy { private readonly Uri _uri; public WebProxy(string uri) { _uri = new Uri(uri); } public ICredentials Credentials { get; set; } public Uri GetProxy(Uri destination) { return _uri; } public bool IsBypassed(Uri host) { return host.IsLoopback; } } private const string URIs = @"" https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.SqlServer' https://www.nuget.org/api/v2/FindPackagesById()?id='EntityFramework.SqlServer' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.SqlServer' https://nuget.org/api/v2/FindPackagesById()?id='EntityFramework.SqlServer' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='EntityFramework.SqlServer' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.InMemory' https://www.nuget.org/api/v2/FindPackagesById()?id='EntityFramework.InMemory' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.InMemory' https://nuget.org/api/v2/FindPackagesById()?id='EntityFramework.InMemory' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='EntityFramework.InMemory' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Kestrel' https://www.nuget.org/api/v2/FindPackagesById()?id='Kestrel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Kestrel' https://nuget.org/api/v2/FindPackagesById()?id='Kestrel' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Kestrel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Cookies' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Cookies' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Cookies' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Cookies' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication.Cookies' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Facebook' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Facebook' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Facebook' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Facebook' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication.Facebook' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Google' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Google' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Google' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Google' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication.Google' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.MicrosoftAccount' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.MicrosoftAccount' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.MicrosoftAccount' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.MicrosoftAccount' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication.MicrosoftAccount' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OpenIdConnect' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OpenIdConnect' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OpenIdConnect' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OpenIdConnect' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication.OpenIdConnect' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Twitter' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Twitter' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Twitter' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.Twitter' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication.Twitter' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Diagnostics' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Entity' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Entity' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Entity' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Entity' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Entity' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity.EntityFramework' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity.EntityFramework' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity.EntityFramework' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity.EntityFramework' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Identity.EntityFramework' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Mvc' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.TagHelpers' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.TagHelpers' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.TagHelpers' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.TagHelpers' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Mvc.TagHelpers' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.IIS' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.IIS' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.IIS' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.IIS' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Server.IIS' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.WebListener' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.WebListener' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.WebListener' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.WebListener' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Server.WebListener' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Session' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Session' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Session' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Session' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Session' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Server' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Server' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Server' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Server' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.SignalR.Server' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.StaticFiles' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.StaticFiles' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.StaticFiles' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.StaticFiles' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.StaticFiles' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Tooling.Razor' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Tooling.Razor' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Tooling.Razor' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Tooling.Razor' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Tooling.Razor' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGenerators.Mvc' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGenerators.Mvc' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGenerators.Mvc' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGenerators.Mvc' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.CodeGenerators.Mvc' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.CommandLine' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.CommandLine' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.CommandLine' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.CommandLine' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Configuration.CommandLine' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.EnvironmentVariables' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.EnvironmentVariables' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.EnvironmentVariables' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.EnvironmentVariables' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Configuration.EnvironmentVariables' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Json' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Json' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Json' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Json' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Configuration.Json' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Console' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Console' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Console' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Console' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Logging.Console' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.IIS' https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Authentication.Facebook/1.0.0-beta6-12600 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Authentication.Google/1.0.0-beta6-12600 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Authentication.OpenIdConnect/1.0.0-beta6-12600 https://www.myget.org/F/aspnetvnext/api/v2/package/Kestrel/1.0.0-beta6-11901 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Authentication.Twitter/1.0.0-beta6-12600 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Authentication.MicrosoftAccount/1.0.0-beta6-12600 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Diagnostics.Entity/7.0.0-beta6-13321 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Identity.EntityFramework/3.0.0-beta6-12644 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Mvc/6.0.0-beta6-14192 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Mvc.TagHelpers/6.0.0-beta6-14192 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Server.WebListener/1.0.0-beta6-12361 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.Configuration.CommandLine/1.0.0-beta6-11520 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.Configuration.EnvironmentVariables/1.0.0-beta6-11520 https://www.myget.org/F/aspnetvnext/api/v2/package/EntityFramework.SqlServer/7.0.0-beta6-13586 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.Configuration.Json/1.0.0-beta6-11520 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Authentication.Cookies/1.0.0-beta6-12600 https://www.myget.org/F/aspnetvnext/api/v2/package/EntityFramework.InMemory/7.0.0-beta6-13586 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Diagnostics/1.0.0-beta6-13321 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.Logging.Console/1.0.0-beta6-11516 https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting' https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.CodeGenerators.Mvc/1.0.0-beta6-12480 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Session/1.0.0-beta6-10599 https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting' https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.SignalR.Server/3.0.0-beta6-12626 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Tooling.Razor/1.0.0-beta6-13550 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.StaticFiles/1.0.0-beta6-12110 https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Hosting' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OAuth' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OAuth' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OAuth' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication.OAuth' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication.OAuth' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.Relational' https://www.nuget.org/api/v2/FindPackagesById()?id='EntityFramework.Relational' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.Relational' https://nuget.org/api/v2/FindPackagesById()?id='EntityFramework.Relational' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='EntityFramework.Relational' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.Kestrel' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.Kestrel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.Kestrel' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Server.Kestrel' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Server.Kestrel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Newtonsoft.Json' https://www.nuget.org/api/v2/FindPackagesById()?id='Newtonsoft.Json' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Newtonsoft.Json' https://nuget.org/api/v2/FindPackagesById()?id='Newtonsoft.Json' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Newtonsoft.Json' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Threading' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading' https://nuget.org/api/v2/FindPackagesById()?id='System.Threading' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Threading' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authorization' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authorization' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authorization' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authorization' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authorization' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Configuration' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Dynamic.Runtime' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Dynamic.Runtime' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Dynamic.Runtime' https://nuget.org/api/v2/FindPackagesById()?id='System.Dynamic.Runtime' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Dynamic.Runtime' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Authentication' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Authentication' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Identity' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Identity' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.ObjectModel' https://www.nuget.org/api/v2/FindPackagesById()?id='System.ObjectModel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.ObjectModel' https://nuget.org/api/v2/FindPackagesById()?id='System.ObjectModel' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.ObjectModel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Cors.Core' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Cors.Core' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Cors.Core' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Cors.Core' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Cors.Core' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing.Algorithms' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing.Algorithms' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing.Algorithms' https://nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing.Algorithms' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Security.Cryptography.Hashing.Algorithms' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.IdentityModel.Protocol.Extensions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.IdentityModel.Protocol.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.IdentityModel.Protocol.Extensions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.IdentityModel.Protocol.Extensions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.IdentityModel.Protocol.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Runtime' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime' https://nuget.org/api/v2/FindPackagesById()?id='System.Runtime' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Runtime' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Headers' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Headers' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Headers' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Headers' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Net.Http.Headers' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.ApiExplorer' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.ApiExplorer' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.ApiExplorer' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.ApiExplorer' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Mvc.ApiExplorer' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Server' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Server' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Server' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Net.Http.Server' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Net.Http.Server' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Extensions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Extensions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Extensions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Mvc.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Razor' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Razor' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Razor' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Mvc.Razor' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Mvc.Razor' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Memory' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Memory' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Memory' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Memory' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Caching.Memory' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Net.Http.WinHttpHandler' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Net.Http.WinHttpHandler' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Net.Http.WinHttpHandler' https://nuget.org/api/v2/FindPackagesById()?id='System.Net.Http.WinHttpHandler' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Net.Http.WinHttpHandler' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.DependencyInjection' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.DependencyInjection' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.DependencyInjection' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.DependencyInjection' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.DependencyInjection' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Collections' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Collections' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Collections' https://nuget.org/api/v2/FindPackagesById()?id='System.Collections' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Collections' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Collections.Specialized' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Collections.Specialized' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Collections.Specialized' https://nuget.org/api/v2/FindPackagesById()?id='System.Collections.Specialized' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Collections.Specialized' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.ComponentModel' https://www.nuget.org/api/v2/FindPackagesById()?id='System.ComponentModel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.ComponentModel' https://nuget.org/api/v2/FindPackagesById()?id='System.ComponentModel' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.ComponentModel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Claims' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Security.Claims' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Claims' https://nuget.org/api/v2/FindPackagesById()?id='System.Security.Claims' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Security.Claims' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.ComponentModel.TypeConverter' https://www.nuget.org/api/v2/FindPackagesById()?id='System.ComponentModel.TypeConverter' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.ComponentModel.TypeConverter' https://nuget.org/api/v2/FindPackagesById()?id='System.ComponentModel.TypeConverter' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.ComponentModel.TypeConverter' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Diagnostics.Debug' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Diagnostics.Debug' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Diagnostics.Debug' https://nuget.org/api/v2/FindPackagesById()?id='System.Diagnostics.Debug' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Diagnostics.Debug' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Diagnostics.Tools' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Diagnostics.Tools' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Diagnostics.Tools' https://nuget.org/api/v2/FindPackagesById()?id='System.Diagnostics.Tools' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Diagnostics.Tools' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Globalization' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Globalization' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Globalization' https://nuget.org/api/v2/FindPackagesById()?id='System.Globalization' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Globalization' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Linq' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Linq' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Linq' https://nuget.org/api/v2/FindPackagesById()?id='System.Linq' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Linq' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Linq.Expressions' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Linq.Expressions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Linq.Expressions' https://nuget.org/api/v2/FindPackagesById()?id='System.Linq.Expressions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Linq.Expressions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Linq.Queryable' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Linq.Queryable' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Linq.Queryable' https://nuget.org/api/v2/FindPackagesById()?id='System.Linq.Queryable' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Linq.Queryable' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Reflection' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection' https://nuget.org/api/v2/FindPackagesById()?id='System.Reflection' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Reflection' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.FileSystemGlobbing' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.FileSystemGlobbing' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.FileSystemGlobbing' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.FileSystemGlobbing' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.FileSystemGlobbing' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Resources.ResourceManager' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Resources.ResourceManager' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Resources.ResourceManager' https://nuget.org/api/v2/FindPackagesById()?id='System.Resources.ResourceManager' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Resources.ResourceManager' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime.Extensions' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Runtime.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime.Extensions' https://nuget.org/api/v2/FindPackagesById()?id='System.Runtime.Extensions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Runtime.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Principal' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Security.Principal' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Principal' https://nuget.org/api/v2/FindPackagesById()?id='System.Security.Principal' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Security.Principal' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Abstractions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Http.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Caching.Abstractions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Caching.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Text.Encoding' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Text.Encoding' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Text.Encoding' https://nuget.org/api/v2/FindPackagesById()?id='System.Text.Encoding' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Text.Encoding' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging.Abstractions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Logging.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.OptionsModel' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.OptionsModel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.OptionsModel' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.OptionsModel' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.OptionsModel' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.Tasks' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Threading.Tasks' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.Tasks' https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Server.IIS/1.0.0-beta6-11976 https://nuget.org/api/v2/FindPackagesById()?id='System.Threading.Tasks' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Threading.Tasks' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Extensions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Extensions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Http.Extensions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Http.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Data.SqlClient' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Data.SqlClient' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Data.SqlClient' https://nuget.org/api/v2/FindPackagesById()?id='System.Data.SqlClient' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Data.SqlClient' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Text.Encoding.CodePages' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Text.Encoding.CodePages' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Text.Encoding.CodePages' https://nuget.org/api/v2/FindPackagesById()?id='System.Text.Encoding.CodePages' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Text.Encoding.CodePages' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.Core' https://www.nuget.org/api/v2/FindPackagesById()?id='EntityFramework.Core' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='EntityFramework.Core' https://nuget.org/api/v2/FindPackagesById()?id='EntityFramework.Core' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='EntityFramework.Core' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.FileProviders.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.FileProviders.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.FileProviders.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.FileProviders.Abstractions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.FileProviders.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor.Runtime' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor.Runtime' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor.Runtime' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor.Runtime' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Razor.Runtime' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Abstractions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Diagnostics.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Hosting.Abstractions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Hosting.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.Thread' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Threading.Thread' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.Thread' https://nuget.org/api/v2/FindPackagesById()?id='System.Threading.Thread' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Threading.Thread' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.RandomNumberGenerator' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.RandomNumberGenerator' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.RandomNumberGenerator' https://nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.RandomNumberGenerator' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Security.Cryptography.RandomNumberGenerator' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.WebEncoders' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Messaging' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Messaging' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Messaging' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.SignalR.Messaging' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.SignalR.Messaging' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.DataProtection' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.DataProtection' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.DataProtection' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.DataProtection' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.DataProtection' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Console' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Console' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Console' https://nuget.org/api/v2/FindPackagesById()?id='System.Console' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Console' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.IO' https://www.nuget.org/api/v2/FindPackagesById()?id='System.IO' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.IO' https://nuget.org/api/v2/FindPackagesById()?id='System.IO' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.IO' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.WebUtilities' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.WebUtilities' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.WebUtilities' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.WebUtilities' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.WebUtilities' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Logging' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Logging' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders.Core' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders.Core' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders.Core' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.WebEncoders.Core' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.WebEncoders.Core' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.CSharp' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.CSharp' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.CSharp' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.CSharp' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.CSharp' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Collections.Concurrent' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Collections.Concurrent' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Collections.Concurrent' https://nuget.org/api/v2/FindPackagesById()?id='System.Collections.Concurrent' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Collections.Concurrent' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Extensions' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Extensions' https://nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Extensions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Reflection.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Emit' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Emit' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Emit' https://nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Emit' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Reflection.Emit' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Emit.ILGeneration' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Emit.ILGeneration' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Emit.ILGeneration' https://nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Emit.ILGeneration' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Reflection.Emit.ILGeneration' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Primitives' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Primitives' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Reflection.Primitives' https://nuget.org/api/v2/FindPackagesById()?id='System.Reflection.Primitives' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Reflection.Primitives' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.ThreadPool' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Threading.ThreadPool' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.ThreadPool' https://nuget.org/api/v2/FindPackagesById()?id='System.Threading.ThreadPool' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Threading.ThreadPool' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.Timer' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Threading.Timer' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Threading.Timer' https://nuget.org/api/v2/FindPackagesById()?id='System.Threading.Timer' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Threading.Timer' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS.Interop' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS.Interop' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS.Interop' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS.Interop' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS.Interop' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Loader.IIS' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.CodeGeneration' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.EntityFramework' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.EntityFramework' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.EntityFramework' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.EntityFramework' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.EntityFramework' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.Templating' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.Templating' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.Templating' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.Templating' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.CodeGeneration.Templating' https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.FileSystemGlobbing/1.0.0-beta6-11549 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Net.Http.WinHttpHandler/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Authentication.OAuth/1.0.0-beta6-12600 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.Configuration/1.0.0-beta6-11520 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Collections.Specialized/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.SignalR.Messaging/3.0.0-beta6-12626 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Security.Claims/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Threading.Tasks/4.0.10-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Security.Cryptography.Hashing.Algorithms/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/System.ObjectModel/4.0.10-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Threading/4.0.10-beta-23021 http://www.myget.org/F/azureadwebstacknightly/api/v2/package/Newtonsoft.Json/6.0.6 https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime.Handles' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Runtime.Handles' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime.Handles' https://nuget.org/api/v2/FindPackagesById()?id='System.Runtime.Handles' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Runtime.Handles' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Abstractions' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Abstractions' https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Configuration.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/package/System.Threading.Thread/4.0.0-beta-23021 http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Configuration.Abstractions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.X509Certificates' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.X509Certificates' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.X509Certificates' https://nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.X509Certificates' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Security.Cryptography.X509Certificates' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Win32.Primitives' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Win32.Primitives' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Win32.Primitives' https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Loader.IIS/1.0.0-beta6-11976 https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.Win32.Primitives' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.Win32.Primitives' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.IO.FileSystem' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById?id='Microsoft.Framework.DependencyInjection'&$skiptoken='Microsoft.Framework.DependencyInjection','1.0.0-beta6-11513' https://www.nuget.org/api/v2/FindPackagesById()?id='System.IO.FileSystem' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.IO.FileSystem' https://nuget.org/api/v2/FindPackagesById()?id='System.IO.FileSystem' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.IO.FileSystem' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Net.Http' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Net.Http' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Net.Http' https://nuget.org/api/v2/FindPackagesById()?id='System.Net.Http' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Net.Http' https://www.myget.org/F/aspnetvnext/api/v2/package/System.Threading.Tasks/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Threading/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.IO.Compression' https://www.nuget.org/api/v2/FindPackagesById()?id='System.IO.Compression' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.IO.Compression' https://nuget.org/api/v2/FindPackagesById()?id='System.IO.Compression' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.IO.Compression' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime.InteropServices' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Runtime.InteropServices' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Runtime.InteropServices' https://nuget.org/api/v2/FindPackagesById()?id='System.Runtime.InteropServices' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Runtime.InteropServices' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Net.Primitives' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Net.Primitives' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Net.Primitives' https://nuget.org/api/v2/FindPackagesById()?id='System.Net.Primitives' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Net.Primitives' https://www.myget.org/F/aspnetvnext/api/v2/package/System.Reflection.Extensions/4.0.0-beta-23021 http://www.myget.org/F/azureadwebstacknightly/api/v2/package/Newtonsoft.Json/6.0.6 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Security.Cryptography.RandomNumberGenerator/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/EntityFramework.Core/7.0.0-beta6-13586 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.AspNet.Razor.Runtime/4.0.0-beta6-11615 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.CodeGeneration.Templating/1.0.0-beta6-12480 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Threading.Timer/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.Encryption' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.Encryption' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.Encryption' https://nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.Encryption' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Security.Cryptography.Encryption' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing' https://www.nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing' https://nuget.org/api/v2/FindPackagesById()?id='System.Security.Cryptography.Hashing' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Security.Cryptography.Hashing' https://www.myget.org/F/aspnetvnext/api/v2/package/System.IO/4.0.10-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/System.IO/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.WebEncoders/1.0.0-beta6-11657 http://www.myget.org/F/azureadwebstacknightly/api/v2/package/Newtonsoft.Json/6.0.6 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Runtime/4.0.20-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/System.Runtime/4.0.0-beta-23021 https://www.myget.org/F/aspnetvnext/api/v2/package/Microsoft.Framework.DependencyInjection/1.0.0-beta6-11523 https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Text.Encoding.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/package/System.Console/4.0.0-beta-23021 https://www.nuget.org/api/v2/FindPackagesById()?id='System.Text.Encoding.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='System.Text.Encoding.Extensions' https://nuget.org/api/v2/FindPackagesById()?id='System.Text.Encoding.Extensions' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='System.Text.Encoding.Extensions' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor' https://www.myget.org/F/aspnetvnext/api/v2/package/System.Text.Encoding.CodePages/4.0.0-beta-23021 https://nuget.org/api/v2/FindPackagesById()?id='Microsoft.AspNet.Razor' http://www.myget.org/F/azureadwebstacknightly/FindPackagesById()?id='Microsoft.AspNet.Razor' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Roslyn' https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Roslyn' https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Roslyn' ""; } } ``` " 2169 area-Serialization Fix serialization for type which has DataMember with special Name values This issue happens only for DCJS and affecting both NetCore and NetNative, due to missing method from recent merging of DCJS. cc: @SGuyGe @shmao @zhenlan 2171 area-Infrastructure When compiling in TFS, using the ECMA key requires setting before dir.props import Because the TFS build targets sets true, sign.targets won't respect the UseECMAKey property. Additionally, because dir.props sets based on the AssemblyKeyType conditional, AssemblyKeyType needs to be set in the project before importing dir.props. 2174 area-Serialization Deserializing xml on linux coreclr x64 throws System.NotSupportedException "When executing the following code on ubuntu 14.04 using coreclr x64 version 1.0.0-beta6-12120 a System.NotSupportedException is thrown. This works fine on windows with same coreclr version ``` c# public class Program { public class Test3 { public string Title {get;set;} } public void Main(string[] args) { string xml = ""Test 12345""; XmlSerializer serializer = new XmlSerializer(typeof(Test3)); StringReader reader = new StringReader(xml); var value = (Test3)serializer.Deserialize(reader); var title = value.Title; } } ``` the stacktrace: ``` System.InvalidOperationException: There is an error in XML document (0, 0). ---> System.NotSupportedException: Specified method is not supported. at System.Xml.Serialization.XmlSerializerImplementation.get_ReadMethods() at System.Xml.Serialization.TempAssembly.InvokeReader(XmlMapping mapping, XmlReader xmlReader, Object events, String encodingStyle) at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, Object events) --- End of inner exception stack trace --- at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, Object events) at System.Xml.Serialization.XmlSerializer.Deserialize(TextReader textReader) at SampleApp.Program.Main(String[] args) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Framework.Runtime.Common.EntryPointExecutor.Execute(Assembly assembly, String[] args, IServiceProvider serviceProvider) at Microsoft.Framework.ApplicationHost.Program.ExecuteMain(DefaultHost host, String applicationName, String[] args) at Microsoft.Framework.ApplicationHost.Program.Main(String[] args) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Framework.Runtime.Common.EntryPointExecutor.Execute(Assembly assembly, String[] args, IServiceProvider serviceProvider) at dnx.host.Bootstrapper.RunAsync(List`1 args, IRuntimeEnvironment env) at dnx.host.RuntimeBootstrapper.ExecuteAsync(String[] args) at dnx.host.RuntimeBootstrapper.Execute(String[] args) System.NotSupportedException: Specified method is not supported. at System.Xml.Serialization.XmlSerializerImplementation.get_ReadMethods() at System.Xml.Serialization.TempAssembly.InvokeReader(XmlMapping mapping, XmlReader xmlReader, Object events, String encodingStyle) at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, Object events) ``` " 2177 area-Serialization Silent Failure: XmlSerializer.Deserialize Method (XmlReader) "I attempted to file this on Connect, but it appears that even though Connect states that it is still accepting issues, it is closed. There is a bug in the current release version (4.5.2) that fails w/o any errors, on the deserialization of a property of a class that is defined as a List. I haven't tried lists of other types, but they are probably affected as well, and this only affects Deserialize, not Serialize. There are 2 classes below, and both serialize properly, in the exact same way. However, the Works class will deserialize properly, while the DoesNotWork class will not. The DoesNotWork class will end up with an Items property that contains a list with zero items, when 1 or more items exist. We discovered this in a property that has code in both the get and set accessors, while all properties that don't have any get or set code work perfectly. ``` C# [Serialize] [XmlRoot(""root"")] public class Works { [XmlElement(""items"")] public List Items { get; set; } } [Serialize] [XmlRoot(""root"")] public class DoesNotWork { private List _items; [XmlElement(""items"")] public List Items { get { return _items; } set { _items = value; } } } ``` " 2189 area-System.Xml Bring XSL-T to CoreFX The [xUnit.net](https://github.com/xunit) unit test framework uses XSL-T to transform its native XML reporting format into other XML (NUnit, xUnit.net v1) formats as well as to generate HTML reports. This strategy works well for desktop runners, but is currently not supported in our [DNX runner](https://github.com/xunit/dnx.xunit) because of the lack of XSL-T support on the platform. This is a [feature request for our DNX runner](https://github.com/xunit/dnx.xunit/issues/42) which we cannot satisfy without some form of XSL-T support. In particular, the code we use today exercises `XslCompiledTransform`: https://github.com/xunit/xunit/tree/05826dcfd3cee97d93ff63201db0296933dd3511/src/xunit.runner.reporters.desktop 2191 area-System.Data Add Sql Impersonation Support SqlClient team needs to add Impersonation support for Project K, but underlying code support does not currently exist. 2192 area-System.Net Update README.md for System.Net.Http.WinHttpHandler Update the README.md for the System.Net.Http.WinHttpHandler library. 2199 area-Meta Review converged configuration APIs ASP.NET has created configuration APIs (TBD: Link). @divega was driving a working group to push these APIs down so that they can become general purpose BCL-ish APIs. We should review the proposal for a converged API set. 2203 area-System.Security Add support for X509 revocation checks on Unix Due to a lack of a stable networking stack, the X509Chain for Unix does not support CRL downloading or OCSP. There are a couple of different parts to be done: - OCSP (Online checks) - CRL fetching (Online checks) - CRL caching (Online/Offline checks) - CRL cache refresh - Using the CRL during chain building 2204 area-System.Security Figure out a strategy for X509VerificationFlags on Unix Because OpenSSL reports back only the first error it finds, the Ignore\* flags can't be used to disregard error codes coming back. Whatever flags cannot be supported should be changed to a PlatformNotSupportedException. 2207 area-System.Security Add support for additional LocalMachine X509Stores on Unix, or throw PlatformNotSupportedException LM\Root, and to a lesser degree LM\CA, are populated from a system directory. The other stores could be put into another path area, if we wanted to support having writable system stores. If writable system stores makes no sense, then other LM* stores should throw a descriptive PlatformNotSupportedException, not a blank NotImplementedException. 2209 area-System.Numerics Support for SSE4 intrinsics by RyuJIT Support for many of the interesting instructions like popcnt (technically SSE4a) could be an interesting addition and prove to be useful to avoid using unmanaged code in certain performance sensitive applications. Many (technically all) of the operations can be emulated in CPU when not available with specific optimizations for the target platform or even have the ability with specially crafted if-then-else optimizations. That would allow to even switch to an entirely different algorithm without any runtime impact (if properly done at the jitting phase). 2210 area-System.Security Consider multi-threading implications on the OpenSSL ERR_get_error queue. OpenSSL compiled with OPENSSL_FIPS seems to have a single static ERR_STATE object for the process. The default appears to be set up for providing the capabilities for stack-per-thread, but OPENSSL_thread_id() appears to return the process ID, not the thread ID. The mainline seems to be served by `OPENSSL_set_id_callback` to something returning a thread ID; and lightup for FIPS could try invoking `int_ERR_lib_init`, which resets the library to the non-FIPS way of error management (though that theoretically has FIPS implications...). 2217 area-System.Text UTF8Encoding should support encoding/decoding of unpaired surrogates "According to RFC 3629 encoding/decoding unmatched surrogates should be disallowed: ""The definition of UTF-8 prohibits encoding character numbers between U+D800 and U+DFFF, which are reserved for use with the UTF-16 encoding form (as surrogate pairs) and do not directly represent characters."" However, this hasn't been followed by real world encoders/decoders. For example, the ECMA-335 standard encodes string arguments of custom attributes using UTF8 and the compilers allowed unpaired surrogates in the attribute argument. Another example is PDB - the file paths in PDB are stored as UTF8 encoded strings and unpaired surrogates are also allowed. The same for values of local string constants (e.g. `const string surrogate = ""\ud800""`). To avoid breaking changes Roslyn needs to allow unpaired surrogates in the above cases and the MetadataReader should also use a variant of UTF8 encoding that is able to decode them. Currently Roslyn has a custom implementation of UTF8 encoder originating from CCI. In general, it seems that pragmatically a UTF16-UTF8 round-tripping is desirable in certain scenarios and UTF8Encoding should support it. I propose to add a constructor to UTF8 Encoding that takes a bool allowUnpairedSurrogates (false by default) that can be used by both Roslyn and MetadataReader. " 2220 area-System.Net [WIP] Refactor HttpClient and MultipartContent with async/await 2221 area-System.Runtime Replacement for unavailable Assembly methods Hi, How is the supported way to deal with the Assembly class? For instance, I have some method calls like Assembly.Location or Assembly.GetCallingAssembly() which I couldn't find a way to bring those calls to CoreFX. Thanks. 2227 area-System.IO FileSystem intermittent test failures Tests for the FileSystem class intermittently fail. The offenders are: | Test Failure | Notes | | --- | --- | | File_WriteAll_all.runTest | | | File_ReadAll_all.runTest | | | File_WriteAllBytes_StrBtA.runTest | | | File_AppendAll_all.runTest | | | File_Copy_str_str_b.runTest | also leaves leftover test directories (only on failure) | | FileInfo_Refresh.runTest | also leaves leftover test directories (only on failure) | | FileInfo_getLastWriteTime | according to #1313 - I detected no failures on my runs | | FileInfo_Create_str | | | DirectoryInfo_Refresh.runTest | also leaves leftover test directories (only on failure) | | DirectoryInfo_Move_str.runTest | also leaves leftover test directories (only on failure) | The source of the failures is the repeated use of the same test file names causing file already in use exceptions. 2241 area-System.Linq Enumerable.ToArray() fails on the results of some uses of Enumerable.Cast() Generally any implementation of `IEnumerable` should be able to produce a `T[]` with a call to `ToArray()`. However, in some cases the `IEnumerable` produced by a call to `Cast()` will in fact be an array of another type that can be converted on the fly by the implementation of `GetEnumerator()`. This works well in many cases and is a valuable optimisation (rather than going through `CastIterator`) but if `ToArray()` is called on it, the result is an `ArrayTypeMismatchException` that would not be expected by the user, especially since code might call `ToArray()` without knowing that `Cast()` had been used, or pass the result of `Cast()` to a method, not knowing that `ToArray()` would be used. In particular this happens if you `Cast()` an array of one `enum` type to another `enum` type. 2246 area-System.Net HttpClient not thread Safe Hi, Recently I noticed HttpClient, FtpClient (maybe SMTPClient I have to check) are not thread safe when Scheme is set https. Since all those classes relies on static member SecurityProtocol in ServicePointManager. We cant have multiple tasks or threads targeting different SSL or TLS version at the same time. I didn't see ServicePointManager in corefx repository located in System.Net namespace. Are there plans to open this piece of code ? 2249 area-System.Data DbDataReader.Close() doesn't unify with System.Data.Common If I write a provider against System.Data.Common on .NET Standard 1.2 and implement `Dispose` appropriately, then I try to use that provider on .NET Framework 4.6 or .NET Core 2.0, the following code won't work. ``` C# reader.Close(); // No-op Assert.True(reader.IsClosed); // Fails since Dispose is never called ``` It fails because on .NET Framework and .NET Core 2.0, `Dispose` calls `Close` (not the other way around) and since there is no virtual `Close` method on .NET Standard 1.2, I cannot override it without cross-targeting the frameworks that do have it. 2252 area-System.Collections Provide a sorting algorithm with good behavior on nearly-sorted inputs. The existing 4.5 Array.sort is implemented with intro sort. While this has reasonable behavior, it effectively has the same number of comparisons regardless of input size. This number of comparisons is (as expected) around `n * log_n`. While this is acceptable for random inputs, it is unfortunately much slower for nearly-sorted inputs. More recent algorithms (like TimSort) can provide massive improvements here, sorting nearly-sorted lists with a nearly linear number of comparisons. For code-paths where sorting is expensive, and where comparisons are not cheap, a better sort algorithm built into the Fx would be appreciated. Whether or not this becomes the default is up to you. But providing it as an option would still be a huge boon. Note: this hits Roslyn+TypeScript. We often produce trees (i.e. for tagging) from nearly sorted data. There may be hundreds of thousands of elements in this data. Being able to sort in near-linear time means we can then build our trees in linear time. With data sets this large, the difference between `n` and `n log n` is actually quite significant. 2254 area-System.Runtime Provide ways to parse sections of a string, not an entire string. Right now .Net provides lots of ways to convert strings to other values. For example `Int32.TryParse`. This is fine if you are starting with a string, but it's very suboptimal if you have a larger string, and are trying to parse out a piece of it. You are forced to first create a substring, and then parse that. This adds GC pressure to the system and can be a large perf impact. It would be very beneficial to have overloads of these functions that took a start+length. Alternatively, if there were some sort of struct representing a piece of a string, that could also work as well. Right now this impacts TypeScript greatly. We have to marshal data from JavaScript to .Net. This comes in as a string to us, which we then need to parse out into actual values (like ints). We are forced to either send tons of strings over (not ideal), or send one big string over that we then are forced to break into tons of small strings. Right now we've worked around this by hand-writing our own Int32.Parse that can work on subsections of a string. But this is non-ideal and would really be served inside the runtime so everyone could benefit. This is a place where really ony the runtime could implement this thing. There is so much complexity around string parsing (even just for ints), with things like FormatSpecifiers and whatnot, that trying to duplicate it in another library would be unfeasible. 2255 area-System.Linq IntersectIterator not optimized to exit early In System/Linq/Enumerable.cs, the `IntersectIterator` method will iterate over all of the items of `first`, even if the `set` created by `second` is empty. A simple performance optimization would be to `yield break` when `set` is empty. 2260 area-System.Net Merge changes from TFS 2273 area-System.Security Enable AuthorityInformationAccess downloads for X509Chain on Unix. Windows will follow the AuthorityInformationAccess record and attempt to download any intermediate certificates as part of building the X509Chain, we should enable this for the Unix platforms as well, to maintain feature parity. **Important**: Downloaded self-signed certificates are NOT considered trustworthy, and they need to make the chain report as not valid (unless root trust was bypassed). Also important: Any testing for this should be [OuterLoop], because it would depend on the network. 2278 area-System.Collections Rename Validation to Microsoft.Internal.Validation "The C# (and possibly VB compiler) will match the top-level namespace `Validation` in `System.Collections.Immutable` even though it only contains internal types. This prevents customers from using a type named `Validation` in their code. Specifically, assume a customer has defined the type like this: ``` C# namespace Support { public static class Validation { public static bool IsValid(string x) { return true; } } } ``` Consuming this type stops working when `System.Collections.Immutable` is referenced: ``` C# using System; using System.Collections.Generic; using System.Linq; using System.Text; using Support; namespace ImmutableTest { public class Experiment { // With out Immutable included, the below works just fine. // Once I include immutable, Validation, which was really // Support.Validation, but was found because of the using // statement above, it starts to fail showing that it can't // find IsValid in the namespace Validation. public void Test1() { Validation.IsValid(""AS""); } public void Test2() { Support.Validation.IsValid(""AS""); } } } ``` According to @jaredpar this behavior is by-design: > This behavior is by design for C# at least. A simple name lookup will prefer matching namespaces over types (section 7.5.2). I didn’t check the VB spec but it almost assuredly has the same design here. In order to avoid that conflict, we should rename the `Validation` namespace to something like `System.Validation`. " 2279 area-System.IO File System Watching for the Root Watch Folder doesn't match on all platforms The FSW on Unix and OS X can receive notifications about the root folder being watcher but the Windows implementation cannot. There are also differences (lock being taken) on Desktop CLR that might not be the case on CoreCLR 2287 area-Serialization DataContractJsonSerializerTests.DCJS_EnumerableCollection test fails in UTC+03.00 (Minsk) timezone "DataContractJsonSerializerTests.DCJS_EnumerableCollection fails in my timezone (UTC+03.00, Minsk). Current code: ``` C# var offsetMinutes = (int)TimeZoneInfo.Local.BaseUtcOffset.TotalMinutes; var timeZoneString = string.Format(""{0:+;-}{1}"", offsetMinutes, new TimeSpan(0, offsetMinutes, 0).ToString(@""hhmm"")); var original = new EnumerableCollection(); original.Add(new DateTime(2000, 1, 1).AddMinutes(offsetMinutes)); ``` Fix: ``` C# var date = new DateTime(2000, 1, 1); var offsetMinutes = (int)TimeZoneInfo.Local.GetUtcOffset(date).TotalMinutes; var timeZoneString = string.Format(""{0:+;-}{1}"", offsetMinutes, new TimeSpan(0, offsetMinutes, 0).ToString(@""hhmm"")); var original = new EnumerableCollection(); original.Add(date.AddMinutes(offsetMinutes)); ``` " 2292 area-Serialization Improve test coverage for XmlSerializer 2295 area-System.Xml System.Xml.XPath to support XPath 2, XPath 3 and their XSLT variants #### Motivation `System.Xml.XPath` currently conforms with XPath 1.0 [[W3C-xpath-1]](http://www.w3.org/TR/xpath/) and XSLT 1.0 [[W3C-xslt-1]](http://www.w3.org/TR/xslt/) standards, but not XPath 2.0 [[W3C-xpath-2]](http://www.w3.org/TR/xpath20/), XPath 3.0 [[W3C-xpath-3]](http://www.w3.org/TR/xpath-30/), XPath 3.1 [[W3C-xpath-3.1]](http://www.w3.org/TR/xpath-31/), XSLT 2.0 [[W3C-xslt-2]](http://www.w3.org/TR/xslt20/) and XSLT 3.0 [[W3C-xslt-3]](http://www.w3.org/TR/xslt-30/). The missing standard implementations in BCL are required by many consumer scenarios, for which .NET applications rely on [third party libraries](http://stackoverflow.com/a/1533114/863980). One of the chief scenario is Content Query Web Part (CQWP) in SharePoint, where the users' XSLT code can be drastically minimized if v2 is supported by `System.Xml.XPath`. As for most parts, there are backward compatibility fallbacks available, that is; the code written in XSLT 2 precisely, can be expressed verbosely in XSLT 1 and since so forth. ### Pitfalls Unfortunately, (besides the existing third-party libraries' APIs) I do not have an off-hand -- concrete -- method list to propose, as it requires further brain-storming on whether to auto-select processor based on the input or to explicitly separate the namespaces (`System.Xml.XPath2` and `System.Xml.XPath3`). The point to ponder being; since the sub-languages XPath 2 and XPath 3 intrinsically facilitates backward compatibility modes, see [XPath 2: J.1.3 Backwards Compatibility Behavior](http://www.w3.org/TR/xslt20/#backwards-compatibility-behavior) and [XPath 3: 3.10 Backwards Compatible Processing](http://www.w3.org/TR/xslt-30/#backwards), should the API be any different than the existing one and let consumers select the standard mode? 2315 area-System.Drawing Primitive Color Type for .NET Core ## Primitive Color Type for .NET Core The previous discussion of primitives showed that color discussion should expand into a separate issue. The primitive discussion was started because many customers wanted to move to the .NET Core, but were blocked by the lack of System.Drawing primitives there. There was much discussion and internal debate over defining Point, Rect, and Size and much of that was fueled by scenarios that you guys cited in the previous primitives issue. (Thank you!) Here we are looking for additional scenarios to discuss the advantages of creating a new Color type for the cross-platform .NET Core Graphics API. Feel free to repost your two cents from the previous primitives issue. Thanks for your input! Topics of Interest: (This is where providing scenarios would be awesome) 1. Naming (ColorRgba, sRGB) 2. Different Color Formats (int8 bits, float 16 bits, float 32 bits, alpha or not) Color4F, Color3F, Color4F 3. Color Spaces (conversion issues) 4. Gamma vs Linear 5. Extension Methods (vector operations: lerp, scaling, conversions: from RGB to HSB and reverse, from vectors) 6. Semantic parity with CSS 7. WinRT Compatibility Limit Scope: KnownColors will not be brought over as they don’t always translate well into the cross-platform world depending on the hardware so let’s focus our discussion on defining the Color object for now. 2316 area-System.Linq LINQ/PLINQ `Sum(int)` does not loft to long for intermediate calculation. Currently, even if the total value of elements in a collection falls within the range of the type, LINQ/PLINQ will throw errors if the _running_ total would overflow. That is: ``` new[] { int.MaxValue, int.MaxValue, int.MinValue }.Sum() ``` ... currently throws an overflow exception because of the first two items, even though the total is within range (`int.MaxValue - 1`). It may be beneficial to loft to long for the intermediate cancellation, similar to how float lofts to double (although that particular one is more about an accurate result, not exactly overflow). 2317 area-System.Linq Make IOrderedEnumerable covariant Today, I was trying to convert an `IOrderedEnumerable` to an `IOrderedEnumerable`, but the compiler responded with an error. Looking into the interface, there doesn't seem to be any reason why it can't be [covariant](https://msdn.microsoft.com/en-us/library/dd799517%28v=vs.110%29.aspx), and in fact when I copied and pasted the interface and changed the class signature to `IOrderedEnumerable` everything compiled fine. Can we change this in the BCL, as well? There doesn't seem to be a reason not to. Sidenote: This **may** cause some [breaking changes](http://stackoverflow.com/a/2843563/4077294), but as the post points out this has already been done with `IEnumerable` on a greater scale. ### Proposed API ```C# // Updates existing interface in being covariant. public interface IOrderedEnumerable : IEnumerable { // Existing member IOrderedEnumerable CreateOrderedEnumerable(Func keySelector, IComparer comparer, bool descending); } ``` *(edited by @terrajobst to include the API write-up from @JonHanna)* 2320 area-System.Reflection (api review) IL Object Model "@terrajobst From the recent [API Review 2015-07-08](https://github.com/dotnet/apireviews/tree/master/2015-07-08-ildasm-and-drawing) **IL Object Model** **Observations** - Implement a custom object model. - use [Mono's Cecil](https://github.com/jbevain/cecil) implementation. **Parsing** - From the byte stream - From the text representation. **Output** - Byte stream - Text Representation The issue here I see is that some instructions have ""pointers"" to data embedded into the assembly bytes. A very common one is `Strings`. So you're going to need an object to represent that so that construction of the output bytes is correct. **IL Labels** These point to data objects that encoded into the assembly. - Strings being a common one. - `switch` uses them as jump targets. A benefit of an IL object model, is that you could get an IL representation of a method, without doing a full compilation. **Targeting** For example Roslyn should ""compile"" to an IL Object model, first. which then permit for further diagnostic analysers (or an optional layer). On the produced IL For example:- A analyser to help minimise Boxing operations. you'll should be able to trickle that information back to source-code level. **Compilation** The last stage would ""Compilation Targets"" for various architectures eg. x86, x64,ARM and maybe even Typescript. This could be supplied as NuGet package. --- Side Note I implemented a partial implementation of an object model for IL [here](https://github.com/AdamSpeight2008/IL.Sarc) " 2321 area-System.Runtime ReadOnlyCollection interfaces. From MSDN: ``` public class ReadOnlyCollection : IList, ICollection, IEnumerable, IEnumerable, IList, ICollection, IReadOnlyList, IReadOnlyCollection ``` Suggested fix: ``` public class ReadOnlyCollection : IEnumerable, IEnumerable, IReadOnlyList, IReadOnlyCollection, ICollection ``` 2323 area-System.Linq Add Enumerable.ToHashSet We have `Enumerable.To*` methods for many common collections. `HashSet` is sorely missing. This feature should be cheap to implement and not be problematic in any way that I can see. Evidence that this feature is sorely missing: https://www.google.com/webhp?complete=1&hl=en&gws_rd=cr,ssl&ei=#complete=1&hl=en&q=.net+tohashset People are writing this method over and over again. 2324 area-System.Runtime Add common helper methods to the String class "The `String` class is missing some helper methods for string manipulation that are commonly needed. Of course it is possible for developers to add extension methods but that should not be required. here, I'll propose a list of methods that I think can be easily understood and are commonly required. The ordering of this list starts with the least controversial in my mind. 1. `bool Contains(this string str, string value, StringComparison comparisonType)`. We have no Contains version that takes a StringComparison. The workaround with `IndexOf` is awkward and feels like magic. 2. `Replace(..., StringComparison)`. Right now, `Replace` always uses `Ordinal`. 3. `string Left(this string str, int count)` as well as `Right`. `Left` is equivalent to `str.Substring(0, count)` but right is `str.Substring(str.Length - count, count)`. 4. `Truncate(int maxLength)`. This is not equivalent to Substring(maxLength) because Substring throws if the string is short. 5. `bool IsValidIndex(this string str, int index)` tests whether the argument can be used to obtain a character from the indexer. That sometimes comes in handy. Also `bool IsValidRange(this string str, int index, int count)`. Both can be useful for `Debug.Assert` assertions. 6. `string TrimPrefix(this string str, string prefix)` as well as `TrimPostfix`. It turns out that when working with externally generated strings (ETL processes) it is very common to need to remove a prefix. For example to get a JIRA issue number as an int you do `int.Parse(""PROJ-2317"".TrimPrefix(""PROJ-""))`. `Replace(..., ""PROJ-"", """")` is not equivalent and semantically wrong. The `TrimStart` method cannot be used here. Further helper methods would be `EnsurePrefix/Postfix`. 7. Some helpers to work on strings that might be null or empty. This comes up _all the time_. ``` public static string EmptyToNull(this string str) { return string.IsNullOrEmpty(str) ? null : str; } public static string NullToEmpty(this string str) { return str ?? string.Empty; } public static bool IsNullOrEmpty(this string str) { return string.IsNullOrEmpty(str); } public static bool IsNullOrWhitespace(this string str) { return string.IsNullOrWhiteSpace(str); } ``` 1. `SplitLines()`. Returns the string split into lines. Need to define how to deal with OS-specific line endings. Probably, this should just be `str.Replace(""\r"", """").Split('\n')` which seems to work for everything. I can see some issues with some of these proposals. They are not perfect but I want to get the discussion going. I feel the BCL has a blind spot here. The list certainly starts out with a few no-brainers. " 2327 area-System.Linq Make Enumerable.Any use the same optimizations that Count has (ICollection checks) `Enumerable.Count` uses `ICollection` and `ICollection` to perform a fast and allocation-free computation. `Enumerable.Any` should do that as well. The fact that `Any` does not check `ICollection` is pretty much the only downside to using `Any` vs. `Count() != 0`. This downside should be eliminated. This missing optimization causes confusion: - http://stackoverflow.com/questions/2989300/interview-question-any-vs-if-length-0-for-testing-if-a-collection-has-e - http://dpatrickcaldwell.blogspot.de/2011/08/enumerableany-vs-enumerablecount.html 2329 area-System.Threading Use a more dependable policy for thread pool thread injection As of .NET 4.5 the thread pool injects one thread per 0.5sec if it thinks more threads are required. This is a problem if the number of required threads suddenly increases. Example: A corporate ASP.NET website is idle at night. In the morning at 8AM 1000 people log on and start working. If the app is using let's say 100 threads starting from 8AM it will take like 50sec to create all of them. Until then there will be serious delays and timeouts. It is possible to construe arbitrarily bad scenarios. **Problem statement:** If thread pool load suddenly increases in IO bound workloads the pool is too slow to respond. This causes throughput and availability disruption. IO bound workloads relying on synchronous IO are common. Sudden workload changes are also common. Sometimes the workload can change due to a problem outside of the developer's control: A web service timing out or a database become slow. Let me stress that this causes service interruption. You can easily repro this yourself. Run a load test on an ASP.NET site with `Thread.Sleep(10000);`. The thread count goes up by 2 each second. Starting and shutting down a thread was benchmarked by me to be around 1ms in total. Threads are not really an expensive resource. The thread pool should be a lot more eager to create and destroy threads. 500ms delay to potentially save 1ms is not a good trade-off. **Easy fix:** I propose lowering the injection delay to 100ms. This reduces the problem given above by 5x. Ideally, the rate would be configurable. The shutdown delay could be lowered from 30s as well. Keeping an idle thread for 30000ms to save 1ms seems excessive. In this ticket I'm not that concerned with retiring threads, though. **Smarter, riskier fix:** The delay could depend on the number of threads in existence, the core count and the perceived pressure on the thread-pool. The injection rate could be: - 0ms delay for up to (ProcessorCount \* 1) threads - 50ms delay for up to (ProcessorCount \* 4) threads - Starting from that a delay of (100ms \* (ThreadCount / ProcessorCount \* someFloatFactor)). Reasoning: The more the CPU is oversubscribed the slower we want to inject. Maybe we need to have a maximum delay of 1sec. Or, the delay must rise sub-linearly (e.g. sqrt). Note, that the Windows Kernel has some thread pool injection heuristics that apply back pressure the more threads are in existence. This seems to work. Source: https://channel9.msdn.com/Shows/Going+Deep/Inside-Windows-8-Pedro-Teixeira-Thread-pool 2332 area-System.Net Improve ServicePointManager.DefaultConnectionLimit "`ServicePointManager.DefaultConnectionLimit` appears to be `2` on .NET 4.5 in non-ASP.NET apps. This is too low for almost any production load. I don't know what this low limit is supposed to accomplish. I understand the presence of _some_ limit as a safety measure but `2` is really never useful. Many developers have trouble with that: https://www.google.com/webhp?complete=1&hl=en&gws_rd=cr,ssl&ei=#complete=1&hl=en&q=site:stackoverflow.com+%22ServicePointManager.DefaultConnectionLimit%22 When somebody asks ""My HTTP requests are unexpectedly slow"" the default answer is ""DefaultConnectionLimit = int.MaxValue"" and this solves the problem in about 80% of the cases. This is strong evidence that the default should be changed. This is an ongoing cause for trouble and surprise. There are three choices that I can think of: 1. Unlimited. ASP.NET has chosen this (`SetAutoConfigLimits`) presumably for good reasons. I think this is the least surprising choice. When a developer says `new WebClient().Download(...)` he expects the request to be sent right away. The dev is responsible for ensuring a sane degree of concurrency. he can use `SemaphoreSlim` or the service point infrastructure for that. 2. A constant factor on the CPU count of the machine (e.g. 10). Pro: This scales up with the size of the machine. Con: It might be a low-CPU (e.g. 2) machine that does little processing but relays it's request to some backend REST service that does the heavy lifting. 100 outstanding requests per CPU would not surprise me at all in such a situation. The benefits of async IO would be throttled. 3. `1`. At least this would make developers very quickly aware that the limit exists and that they need to change it. Maybe the value of `2` follows from that reasoning. I consider this to be a weak choice that is going to be perceived as stupid (as is `2`). Another reason for ""Unlimited"" is that there should not be global limits but per-service limits. A .NET process often hosts many components each of which potentially take from the global limit. They could interact in surprising (and spooky) ways. I think the .NET Framework should pick the default value that has the least surprise and set this to `int.MaxValue`. Also, consider revising the default value of `Expect100Continue`. I believe there is not a single developer in the world who expects each `POST` request to be performed twice by default. I do not know a single case where this was ever deemed useful. Disabling this behavior would entail, I believe, low compatibility risk. (See http://stackoverflow.com/search?q=Expect100Continue) " 2335 area-System.Collections Improve ConcurrentBag GetEnumerator performance Currently `ConcurrentBag` `GetEnumerator()` causes list copying everytime enumerator is accessed eg. through `foreach`. This causes strain on the GC and list copying is also heavy operation on large bags. There should be alternative methods to enumarate the bag contents or implement `GetEnumerator()` so that it does not cause list copying. I have presented here 3 possbile solutions to this problem. **Solution 1)** Add `ForEach` method to `ConcurrentBag`: ``` CSharp public void ForEach(Func action) { // Short path if the bag is empty if (_headList == null) return; bool lockTaken = false; try { FreezeBag(ref lockTaken); ThreadLocalList currentList = _headList; while (currentList != null) // Go through thread lists { Node currentNode = currentList._head; while (currentNode != null) // Go through thead specific linked list { if (!action(currentNode._value)) return; // User wanted to break the operation currentNode = currentNode._next; } currentList = currentList._nextList; } } finally { UnfreezeBag(lockTaken); } } ``` `ForEach` function takes user specified `Func` delegate. This delegate takes the bag item as an argument and returns true/false depending on whether the user wants to continue or stop the enumeration operation. Here `ForEach` operation would lock the bag so that other threads would need to wait for the operation to finish. Therefore results of this operation would represent the bag as a moment-in-time snapshot. **Solution 2)** Add `GetEnumerableSlim` function: ``` CSharp public IEnumerable GetEnumerableSlim() { ThreadLocalList currentList = _headList; while (currentList != null) { Node currentNode = currentList._head; while (currentNode != null) { yield return currentNode._value; currentNode = currentNode._next; } currentList = currentList._nextList; } } ``` This would require an additional change to the internal class `Node`: ``` CSharp internal class Node { public Node(T value) { _value = value; } public readonly T _value; public volatile Node _next; // This needs to be marked as volatile public Node _prev; } ``` `GetEnumerableSlim` would enumerate the bag contents concurrently without locking the bag. The proposed implementation is thread safe from memory model point of view. The fields `ConcurrentBag._headList`, `ThreadLocalList._head` and `Node._next` are all volatile variables. Therefore all the threads always have the newest value available. In addition the `Node._value` reference/value is immutable. `GetEnumerableSlim` is safe to use concurrently. Of course here the enumerator would miss some of the concurrent additions/removals from the bag. Also enumerator could contain modification after the `GetEnumerator` was called. Similiar behaviour is also used in `ConcurrentDictionary`, `GetEnumerator` which may also miss some of the concurrent changes or contain new changes after the `GetEnumerator` was called. Therefore the `GetEnumerableSlim` is consistent with other concurrent collections and their enumerator implementations. (It would be intresting to know why `ConcurrentBag` `GetEnumerator` was originally implementeted so that it copies the bag contents to a list where as the other concurrent collections enumerate the contents concurrently eg. `ConcurrentQueue`.) **Solution 3)** Replace `GetEnumerator` with `GetEnumeratorSlim` implementation. This solution would totally eliminate the risk of performance degradation due to list allocations. But this might not be possible if the existing code relies on the fact that `GetEnumerator` returns moment-in-time representation of the bag. 2336 area-System.Globalization Rename tests in System.Globalization About 40-50% of the tests in System.Globalization do not have self descriptive names. They are currently named PosTest, NegTest etc. 2337 area-System.Globalization Add Unicode Tests to System.Globalization Write Unicode Tests that are generic and not coupled with a particular version of Unicode. Specifically CharUnicodeInfo.GetNumericValue() and CharUnicodeInfo.GetUnicodeCategory() 2338 area-System.Collections Add ConcurrentBag and ConcurrentQueue Clear method Currently `ConcurrentBag` does not support clearing of the bag. Currently only way to clear the bag is to call `TryTake` in a loop which is suboptimal in high concurrency applications: ``` CSharp T dummy; while(bag.TryTake(out dummy)) {} ``` Instead `Clear` method could be implemented so that the bag is emptied at once. Here is the proposed implementation: ``` CSharp public void Clear() { // Short path if the bag is empty if (_headList == null) return; bool lockTaken = false; try { FreezeBag(ref lockTaken); ThreadLocalList currentList = _headList; while (currentList != null) { currentList.Reset(); currentList = currentList._nextList; } } finally { UnfreezeBag(lockTaken); } } ``` Internal class `ThreadLocalList` needs new `Reset/Invalidate` method which sets the thread specific linked list _head and _tail to null as well as resets the counters. GC would then collect the non referenced linked list nodes. ``` CSharp internal void Reset() { _head = null; _tail = null; _count = 0; _stealCount = 0; } ``` `Clear` would lock the bag for the duration of the operation since it could cause undefined behaviour if other threads would access the bag while linked lists are being reseted. (Similarly ConcurrentDictionary also locks the dictionary when `Clear` is called.) 2339 area-System.IO Finish rewriting System.IO.FileSystem tests Finish rewriting the tests for System.IO.FileSystem and replacing the giant runTest() methods with individual [Fact]s and [Theory]s, replacing every instance of Console.WriteLine along the way. 2342 area-System.Xml XmlSchema in CoreFX Hi, Are there any plans to include XmlSchema in CoreFX? If not, is there any recommended way to manipulate schemas? Thank you. 2352 area-System.Runtime API Addition: Add generic Array.Reverse Add generic `Array.Reverse`. ## Rationale [`Array.Reverse`](https://github.com/dotnet/coreclr/blob/ef1e2ab328087c61a6878c1e84f4fc5d710aebce/src/mscorlib/src/System/Array.cs#L1570-L1606) is non-generic. It includes a fast path for a known set of primitive types via a call into the runtime (see [`TrySZReverse`](https://github.com/dotnet/coreclr/blob/4cf8a6b082d9bb1789facd996d8265d3908757b2/src/classlibnative/bcltype/arrayhelpers.cpp#L390-L424)), otherwise it falls back to slower code paths that involve boxing for value types. This is a significant performance cost for non-primitive value types, ~22x slower. Callers of `Array.Reverse` (like `List.Reverse` and `ImmutableArray.Builder.Reverse`) are affected by this performance issue. As a workaround, [`List.Reverse`](https://github.com/dotnet/coreclr/pull/1231) and [`ImmutableArray.Builder.Reverse`](https://github.com/dotnet/corefx/pull/2354) are being updated to not use the non-generic `Array.Reverse` method until the generic `Array.Reverse` method becomes available. Other non-generic methods on `Array` have generic counterparts like `Array.Sort` and `Array.Sort`, so this would just be making `Reverse` consistent with those. ## Proposed API ``` c# public abstract class Array : ... { // Proposed public static void Reverse(T[] array); public static void Reverse(T[] array, int index, int length); // Existing public static void Reverse(Array array); public static void Reverse(Array array, int index, int length); } ``` 2358 area-System.Net Initial commit of System.Net.WebHeaderCollection Just what it says on the tin. 2368 area-System.Net Address PR feedback for System.Net.WebHeaderCollection - Replace `HeaderParser` with `Func` - Make static fields `readonly` - Optimize `HeaderInfoTable` for allocation volume and remove the static constructor - Replace non-generic collections with generic collections 2369 area-System.IO Directory AccessTime, WriteTime, and CreationTime for nonexistent directories inconsistent "Labeld in code as ""//See VSWhidbey # 92050"" On Windows, Directory.GetAccessTime, Directory.GetWriteTime, and Directory.GetCreationTime (as well as their DirectoryInfo clones) will return default values when queried with a path to a nonexistent directory. On Unix, running those same methods for AccessTime and WriteTime throws a ""FileNotFoundException"". Is this behavioral difference desired? Example: ``` Assert.Equal(DateTime.FromFileTime(0).Ticks, Directory.GetLastAccessTime(""doesntexist"").Ticks); //true on Windows, throws on Ubuntu Assert.Equal(DateTime.FromFileTime(0).Ticks, Directory.GetLastWriteTime(""doesntexist"").Ticks); //same as above ``` " 2370 area-System.Net Move some System.Net.WebHeaderCollection tests from [Fact]s to [Theory]s The current version of xUnit used by CoreFX has problems writing text containing certain code points to the test log. There are tests in S.N.WHC that use these code points. These tests are currently [Fact]s, but should be combined with the appropriate [Theory] once CoreFX has taken an updated version of xUnit that fixes the issue. 2372 area-System.Net Add System.Net.WebHeaderCollection to README.md Add System.Net.WebHeaderCollection to README.md 2375 area-System.Reflection Add metadata tokens/module versions back to reflection libraries in CORE CLR. Recent versions of the CORE clr have removed metadata from the reflection libraries. We are writing .NET diagnostics tools, and these APIs are important for supporting our diagnostics scenarios. We would like the APIs to be added back. 2376 area-System.Numerics BigRational The BCL project included a BigRational type: http://bcl.codeplex.com/releases/view/42782 The code seems to be released under the [Microsoft Public License](http://bcl.codeplex.com/license). Would there be any license related issue around including BigRational in corefx? It looks like it's also available on [NuGet](https://www.nuget.org/packages/BigRationalLibrary/). There's also a Rational type in [Solver Foundation](https://msdn.microsoft.com/en-us/library/microsoft.solverfoundation.common.rational%28v=vs.93%29.aspx), but the code doesn't appear to be freely available. 2380 area-System.Net Address PR feedback for System.Net.WebHeaderCollection Fixes #2368 2382 area-System.Net Initial commit of System.Net.WebSockets 2383 area-Infrastructure Figure out how to provide parameters to unit tests at build/run-time. The System.Net tests are likely to need this in order to provide some configurability for the servers used during testing. 2384 area-System.Net WebException: update WebException.GetStatusFromException after System.Net.Http API changes For now, we use the .HResult of the exception to help us map to a suitable WebExceptionStatus enum value. The .HResult is set into this exception by the underlying .NET Core and .NET Native versions of the System.Net.Http stack. In the future, the HttpRequestException will have its own .Status property that is an enum type that is more compatible directly with the WebExceptionStatus enum. 2385 area-System.Net System.Net.Requests.Tests: HttpWebResponse tests need a server that does not provide a Content-Type header Just what it says on the tin. The httpbin servers we're currently using for testing do not have this characteristic. 2386 area-System.Net Initial commit of System.Net.Requests This change also contains some minor refactorings in System.Net.{Http,WebHeaderCollection} in order to share some code between the various System.Net assemblies. 2387 area-System.Net Refactor Http tests for multiple test projects "This change sets up the layout for having multiple test projects for Http. It follows the current guidlines as well as working within the BUILDTOOLS requirements of having "".Tests.csproj"" for the names of the projects. After this PR is done, we will be migrating a large set of http functional tests to GitHub " 2390 area-System.IO FileSystem test update - Updated Directory tests. Note that there are still some unresolved issues pending a solution to #1221 - Consolidated AccessTime, WriteTime, and CreationTime tests for FileInfo, File, DirectoryInfo, and Directory into one file per folder - GetSetTimes.cs - Removed failing tests for the above properties - Added some Unix tests - Resolves #2359 2392 area-System.Collections Add ImmutableArray.Builder.AddRange, MoveToArray methods The builder currently has APIs: ``` C# public void AddRange(T[] items, int length); public ImmutableArray MoveToImmutable(); ``` but is missing the following: ``` C# public void AddRange(T[] items, int start, int length); public T[] MoveToArray(); ``` 2394 area-Meta Port Workflow Foundation to .NET Core Hello, I don't see in the plans neither here and coreCLR for porting Workflow Foundation for CoreCLR... We want know how can we start porting it and add PRs for it here. Thanks 2399 area-System.Net Address CR feedback for System.Net.WebSockets. - Document the ASCII validation in `WebSocketValidate.ValidateSubProtocol` - Clean up `using` statements in `WebSocketValidate` - Decide which exception type to derive from in `WebSocketException` 2400 area-System.Linq Take advantage of algorihmic differences between first-from-ordered and ordered. The fact of something being ordered can have implications for subsequent operations. The ordering Linq methods (`OrderBy`, `ThenBy`, etc.) operate first in O(1) time and space to make a promise of a subsequent O(n log n) sort in O(n) space. In many case though the following operation does not need all of the promised operation. `.OrderBy(λ).First()`, `.OrderBy(λ).First(λ)`, and so on request that a sequence be sorted (O(n log n) time/O(n) space) and then the first element taken (O(1) time and space) which is in total an O(n log n) time/O(n) space operation. However, it is reducible to the O(n) time/O(1) space operation of identifying just the item in question which can be done in O(n) time and O(1) space. As `First`, `Last` and their variants are particularly common with ordered sequences (in some cases not having any meaning without an order), I think this would be a worth-while optimisation. There are other optimisations available, such as all aggregate operations (`Sum` etc.) could be moved from O(n log n) time to O(n) time simply by extracting the source and ignoring the ordering, this may not be as common a case as to make it worthwhile. One could argue that the user should have not put the ordering in there in the first place, and while the cost of checking for an ordered source is slightly, it's not worth paying it on all aggregate operations to benefit a flawed case. `Reverse` is somewhere in the middle. It's reasonable to expect someone might construct an ordered sequence and then `Reverse` in some cases. It's got subtleties (one cannot simply reverse all `OrderBy` to `OrderByDescending` and vice-versa because `OrderBy` is documented as performing stable ordering in linq-2-objects and therefore reversed must have reverse source ordering on equivalent items) but wouldn't be hard, but it's less clear whether the pay off would be worth it. In the more useful cases of `First` etc. there is a fly in the ointment in that this would affect the number of times both key selectors for ordering and predicates for `First` etc. were called. As per #2349 this means side effects on `Func`s would be affected, though perhaps in a way that was less predictable to the user than with that issue anyway. As such, changing the behaviour of `First` in this case would require either a decision in favour of the change that issue would allow for, or a decision that this case was less harmed by such changing side-effects. In summary. I propose that `OrderBy(…).First(…)` and its cousins be optimised, and the trade-offs of optimising `OrderBy(…).Reverse()` be considered. 2402 area-System.IO DirectoryInfo.CreateSubdirectory(path) trims whitespace and returns parent directory "Calling directory.CreateSubdirectory(""whitespaces"") will return ""directory"" and create no subdirectory. ``` DirectoryInfo parentDir = new DirectoryInfo(TestDirectory); DirectoryInfo childDir = parentDir.CreateSubdirectory("" ""); Assert.Equal(parentDir.FullName + Path.DirectorySeparatorChar, childDir.FullName); ``` If newlines or tabs are used, then it throws as expected: ``` DirectoryInfo parentDir = new DirectoryInfo(TestDirectory); Assert.Throws(() => parentDir.CreateSubdirectory(""\n"")); Assert.Throws(() => parentDir.CreateSubdirectory(""\t"")); ``` The reason for this is that dir.CreateSubdirectory(path) essentially wraps around Directory.CreateDirectory(Path.Combine(dir.FullName, path)) which trims trailing whitespace (in this case ""path""). Is this functionality desired? It makes sense to keep curtailing trailing whitespace in Directory.CreateDirectory for back compat reasons, but having CreateSubdirectory on whitespace return the parent directory is unintuitive. " 2403 area-System.IO DirectoryInfo.CreateSubdirectory(path) throws ArgumentException for newline and tab nonsignificant trailing characters "There's a discrepancy in the way DirectoryInfo.CreateSubdirectory and Directory.CreateDirectory handle trailing tab and newline characters on Windows. The below code exhibits the difference: ``` { // Directory.CreateDirectory curtails trailing tab and newline characters string filePath = IOServices.RemoveTrailingSlash(GetTestFilePath()); DirectoryInfo result = Directory.CreateDirectory(filePath + ""\n""); Assert.Equal(filePath, IOServices.RemoveTrailingSlash(result.FullName)); } { // DirectoryInfo.CreateSubdirectory throws an ArgumentException when there are newline or tab characters anywhere in a file name, even as trailing chars string fileName = IOServices.RemoveTrailingSlash(GetTestFileName()); DirectoryInfo parentDir = new DirectoryInfo(TestDirectory); Assert.Throws(() => parentDir.CreateSubdirectory(fileName + ""\n"")); //illegal characters in path } ``` This is somewhat related to #2402 in that DirectoryInfo.CreateDirectory handles whitespace very differently than Directory.CreateDirectory or even DirectoryInfo.Create. " 2405 area-System.Net Refactor WinHttpHandler tests for multiple test projects "This change sets up the layout for having multiple test projects for WinHttpHandler. It follows the current guidlines as well as working within the BUILDTOOLS requirements of having "".Tests.csproj"" for the names of the projects. Also, fixed up some cross-platform definitions in the solutions files for Http and WinHttpHandler. After this PR is done, we will be migrating a large set of http functional tests to GitHub " 2408 area-System.Net Fix WebHeaderCollection.GetEnumerator(). Change WebHeaderCollection.GetEnumerator() from an explicit implementation to a public override to avoid delegating to the base collection. This change also adds a test for this behavior. 2409 area-System.Net Address PR feedback for System.Net.Requests - Audit WebException.cs for inefficient usage of string.Format - Audit the code for string.Compare vs. string.Equals 2416 area-System.Net Abort_BeginGetRequestStreamThenAbort_RequestStreamCallbackCalledBeforeAbortReturns failed in CI "http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug/813/consoleFull ``` System.Net.Requests.Test.HttpWebRequestTest.Abort_BeginGetRequestStreamThenAbort_RequestStreamCallbackCalledBeforeAbortReturns(remoteServer: Uri { AbsolutePath = ""/post"", AbsoluteUri = ""http://httpbin.org/post"", Authority = ""httpbin.org"", DnsSafeHost = ""httpbin.org"", Fragment = """", ... }) [FAIL] 19:58:58 Assert.Equal() Failure 19:58:58 Expected: 1 19:58:58 Actual: 0 19:58:58 Stack Trace: d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Requests\tests\HttpWebRequestTest.cs(548,0): at System.Net.Requests.Test.HttpWebRequestTest.Abort_BeginGetRequestStreamThenAbort_RequestStreamCallbackCalledBeforeAbortReturns(Uri remoteServer) ``` " 2424 area-System.Net Remove unstable System.Net.Requests tests As per issue #2416 remove unstable System.Net.Requests tests that were inadvertently ported to GitHub. 2427 area-Infrastructure Update CoreFx build to shipping nuget Need to do the following to bring ourselves current with the shipping nuget scenarios. - [x] switch from DNX to nuget.exe - [x] move projects from `dnxcore50` to `netstandard1.x` - [ ] use PCL meta-package in all test projects - [x] eliminate `test-runtime\project.json` hack - [ ] ensure we use inbox nuget task/targets when present and only use BuildTools version when inbox is missing - [x] switch to stable version consumption of packages 2433 area-System.IO Update to FileSystem Tests - Consolidated AccessTime, WriteTime, and CreationTime tests for FileInfo, File, DirectoryInfo, and Directory into one file per folder - GetSetTimes.cs - Added some Unix tests - Overhauled the Write/Read/Append tests for the File class and removed many duplicate tests - Continuation of #2390 2435 area-Serialization DataContractSerializer constructor causes NullRefException when serializing object When using [this](https://github.com/dotnet/corefx/blob/6e5d09b0334ca8735962f29f38a390c1298639c0/src/System.Private.DataContractSerialization/src/System/Runtime/Serialization/DataContractSerializer.cs#L51) constructor for the DataContractSerializer: ``` csharp public DataContractSerializer(Type type, string rootName, string rootNamespace, IEnumerable knownTypes) ``` The resulting serializer can throw a `NullReferenceException`. This is caused by the creation of `XmlDictionaryString` objects which have a `null` `XmlDictionary`. There's also a bug/change in behavior in `XmlDictionaryString` compared with Desktop. On Desktop, if you specify a `null` value for the dictionary when constructing a new instance, an `ArgumentNullException` will be thrown. In corefx, it will attempt to create an empty instance of `IXmlDictionary` and assign that. It then straight away assigns the passed in `null` value to the dictionary effectively throwing away the empty dictionary that was created. Some of the Xml writers require the dictionary for an `XmlDicationary` string to be non-null. This is causing a `NullRefException` to be thrown. Here is a partial call stack from WCF: ``` csharp at System.Xml.XmlBinaryWriterSession.PriorityDictionary`2.TryGetValue(K key, V& value) at System.Xml.XmlBinaryWriterSession.TryLookup(XmlDictionaryString s, Int32& key) at System.Xml.XmlBinaryNodeWriter.TryGetKey(XmlDictionaryString s, Int32& key) at System.Xml.XmlBinaryNodeWriter.WriteXmlnsAttribute(String prefix, XmlDictionaryString ns) at System.Xml.XmlBaseWriter.NamespaceManager.DeclareNamespaces(XmlNodeWriter writer) at System.Xml.XmlBaseWriter.EndStartElement() at System.Xml.XmlBaseWriter.AutoComplete(WriteState writeState) at System.Xml.XmlBaseWriter.StartElement(String& prefix, String localName, String ns, XmlDictionaryString xNs) at System.Xml.XmlBaseWriter.WriteStartElement(String prefix, XmlDictionaryString localName, XmlDictionaryString namespaceUri) at System.Runtime.Serialization.XmlWriterDelegator.WriteStartElementPrimitive(XmlDictionaryString localName, XmlDictionaryString namespaceUri) at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteString(XmlWriterDelegator xmlWriter, String value, XmlDictionaryString name, XmlDictionaryString ns) at WriteFaultDetailToXml(XmlWriterDelegator , Object , XmlObjectSerializerWriteContext , ClassDataContract ) at System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator xmlWriter, Object obj, XmlObjectSerializerWriteContext context) at System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle) at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle declaredTypeHandle) at System.Runtime.Serialization.DataContractSerializer.InternalWriteObjectContent(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver) at System.Runtime.Serialization.DataContractSerializer.InternalWriteObject(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver) at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver) at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator writer, Object graph) at System.Runtime.Serialization.XmlObjectSerializer.WriteObject(XmlDictionaryWriter writer, Object graph) ``` 2444 area-System.Data Open items for System.Data.Common The following needs to be incorporated based on the comments in the pull request https://github.com/dotnet/corefx/pull/2197 Please note that the code is being submitted via another pull request https://github.com/dotnet/corefx/pull/2443 1. Investigate the CancellationToken for the Tasks 2. There are more comments made by Stephen Toub, which need to be taken care of. Refer to https://github.com/dotnet/corefx/pull/2197 2454 area-System.Threading Remove reentrancy by default from the TPL The TPL has a few performance features that cause reentrancy. Here are the ones I know of: 1. `TaskCompletionSource.SetXxx` can cause continuations to be executed synchronously. AFAIK this can happen both with or without `ExecuteSynchronously`. Starting with .NET 4.6 there is `RunContinuationsAsynchronously` to disable this behavior. 2. `Task.Wait()` and `Task.Result` can execute an unstarted CPU-based task inline. I do not know of any good workaround from the perspective of the caller of `Wait`/`Result`. Probably, `static Task.Wait(Task)` and related methods have the same problem. 3. `CancellationTokenSource.Cancel()` can execute registered notifications inline. 4. `TaskAwaiter` uses `TaskContinuationOptions.ExecuteSynchronously` if there is no synchronization context set. This forces reentrancy for whatever code completes the task being awaited. In all of these situations arbitrary code can run as part of invoking the respective TPL method. You might not control that code. There might be no way for you to know what code that is because you are writing a library. In big apps it might be architecturally impossible/unwanted to know what code might run. This is **arbitrary, non-deterministic reentrancy in multi-threaded code**. This is very dangerous behavior. Concrete problems (examples): 1. You might call `TaskCompletionSource.SetResult()` under a lock. Now, arbitrary code might run under your lock as well. This can **deadlock**. 2. That arbitrary code might perform blocking operations (e.g. IO). This will now run under your lock and **extend the lock hold time**. 3. The **invariants** of your data might be **temporarily broken**. Due to reentrancy other code can now observe invalid data. 4. **Monitor might fail to provide mutual exclusion** because `lock (x) { lock (x) { } }` does not block/deadlock. Inlining code onto the current thread opens the gates for any locks held by this thread. 5. You might run on a thread with **thread-dependent state** (STA thread, changed priority, deviating culture, `HttpContext.Current` null or not null, ...). This state is now exposed to arbitrary code. It is made available for mutation as well. **This is very unsafe behavior by default.** The number of questions that come up about it on Stack Overflow is noticeable. The bugs are insidious (racy). Please disable _all_ of these behaviors by default. Make unsafe performance features opt in. I doubt there will be compatibility impact from doing this because all of the behaviors above are, to my knowledge, non-deterministic. Completion callbacks should always be queued to the thread pool (or to any scheduler if specified) by default. The TPL should be safe by default. Reentrancy behavior should be: 1. Off by default. 2. Opt-in possible. 3. Overriding forced opt out by any side taking part (completing thread and registering thread). A library should be able to force-disable any of this, no matter whether it registers continuations or whether it completes tasks. This is a problem both for monolithic apps as well as for library authors. See this high-profile case that lead to the inclusion of `RunContinuationsAsynchronously` in .NET 4.6: http://stackoverflow.com/questions/22579206/how-can-i-prevent-synchronous-continuations-on-a-task 2458 area-System.IO FileInfo.Open() with FileMode.Append throws unexpected exception The following code is an example of the situation: ``` string fileName = GetTestFilePath(); using (FileStream fs = new FileInfo(fileName).Open(FileMode.Create)) { fs.WriteByte(0); } using (FileStream fs = new FileInfo(fileName).Open(FileMode.Append)) // Throws ArgumentException: append access can be requested only in write-only mode { } ``` The issue is a result of the default FileAccess value used in the FileInfo.Open(FileMode) method: ``` public FileStream Open(FileMode mode) { return Open(mode, FileAccess.ReadWrite, FileShare.None); } ``` when it should be the same as what File.Open(FileMode) does like so: ``` public FileStream Open(FileMode mode) { return Open(mode, (mode == FileMode.Append ? FileAccess.Write : FileAccess.ReadWrite), FileShare.None); } ``` 2459 area-System.IO DriveLetter_Unix currently failing System.IO.FileSystem.Tests.Directory_CreateDirectory.DriveLetter_Unix [FAIL] 09:58:09 System.IO.DirectoryNotFoundException : Could not find a part of the path '/mnt/j/workspace/dotnet_corefx_linux_debug_tst_prtest/bin/tests/Windows_NT.AnyCPU.Debug/System.IO.FileSystem.Tests/dnxcore50/C:'. 09:58:09 Stack Trace: 09:58:10 at System.IO.UnixFileSystem.RemoveDirectoryInternal(String fullPath, Boolean recursive, Boolean throwOnTopLevelDirectoryNotFound) 09:58:10 at System.IO.UnixFileSystem.RemoveDirectory(String fullPath, Boolean recursive) 09:58:10 at System.IO.Directory.Delete(String path) 09:58:10 at System.IO.FileSystem.Tests.Directory_CreateDirectory.DriveLetter_Unix() 09:58:10 Finished: System.IO.FileSystem.Tests 2460 area-System.IO File.Move relies on the UnixFileSystem CopyFile method that overwrites only when it shouldn't The File.Move(source, dest) and FileInfo.MoveTo(dest) functions do not currently function as they should with regards to overwrites. The following test passes on Windows, but fails on Linux: ``` FileInfo testFileSource = new FileInfo(GetTestFilePath()); testFileSource.Create().Dispose(); FileInfo testFileDest = new FileInfo(GetTestFilePath()); testFileDest.Create().Dispose(); Assert.Throws(() => File.Move(testFileSource.FullName, testFileDest.FullName)); // doesn't throw on Ubuntu ``` 2461 area-System.Data [RC1 blocker] Windows interop getting called on OS X Using SqlClient on CoreCLR (Darwin) calls interop methods that require Windows APIs. This is a regression. This behavior just showed after upgrading DNX. Currently using `dnx-coreclr-darwin-x64.1.0.0-beta7-12264`. Here is the stack trace: > ``` > System.DllNotFoundException : Unable to load DLL 'api-ms-win-core-processthreads-l1-1-0.dll': The specified module could not be found. > (Exception from HRESULT: 0x8007007E) > Stack Trace: > at Interop.mincore.GetCurrentThread() > at Interop.mincore.OpenThreadToken(TokenAccessLevels desiredAccess, WinSecurityContext openAs, SafeAccessTokenHandle& tokenHandle) > at System.Security.Principal.WindowsIdentity.GetCurrentToken(TokenAccessLevels desiredAccess, Boolean threadOnly, Boolean& isImpersonating, Int32& hr) > at System.Security.Principal.WindowsIdentity.GetCurrentInternal(TokenAccessLevels desiredAccess, Boolean threadOnly) > at System.Data.ProviderBase.DbConnectionPoolIdentity.GetCurrent() > at System.Data.ProviderBase.DbConnectionPoolGroup.GetConnectionPool(DbConnectionFactory connectionFactory) > at System.Data.ProviderBase.DbConnectionFactory.GetConnectionPool(DbConnection owningObject, DbConnectionPoolGroup connectionPoolGroup) > at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) > at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) > at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) > at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) > at System.Data.SqlClient.SqlConnection.Open() > at Microsoft.Data.Entity.SqlServer.FunctionalTests.SqlServerTestStore.DeleteDatabase(String name) > at Microsoft.Data.Entity.SqlServer.FunctionalTests.SqlServerTestStore.CreateTransient(Boolean createDatabase) > at Microsoft.Data.Entity.SqlServer.FunctionalTests.SqlServerTestStore.CreateScratch(Boolean createDatabase) > at Microsoft.Data.Entity.FunctionalTests.SharedCrossStoreFixture.CreateTestStore(Type testStoreType) > at Microsoft.Data.Entity.FunctionalTests.SqlServerCrossStoreFixture.CreateTestStore(Type testStoreType) > at Microsoft.Data.Entity.FunctionalTests.EndToEndTest`2..ctor(TFixture fixture) > at Microsoft.Data.Entity.FunctionalTests.SqlServerEndToEndTest..ctor(SqlServerCrossStoreFixture fixture) > ``` 2463 area-System.Net Fix proxy authentication handling during redirects /cc @CIPop @pgavlin @SidharthNabar @ChengTian Reference issue #2168 2467 area-System.Net Avoid internal enumerator allocations in HttpClient There are many places where `ObjectCollection` instances are enumerated internally. Each time one of these collections is enumerated, an enumerator is allocated on the heap. This change avoids the enumerator allocation by using a struct enumerator (internal-only). 2470 area-System.Net Avoid a closure/delegate allocation in WinHttpHandler One-line change to avoid a closure and delegate allocation on each call. 2473 area-System.Net Initial xplat work for HttpClient Created a CurlHandler class to wrap calls to libcurl. Very limited support currently which will be extended in future checkins. Start of #2480 2480 area-System.Net Implement System.Net.Http on Unix 2481 area-System.Net Implement System.Net.Primitives on Unix 2482 area-System.Net Implement System.Net.Sockets on Unix 2483 area-System.Net Implement System.Net.Security on Unix 2484 area-System.Net Implement System.Net.NameResolution on Unix 2485 area-System.Net Implement System.Net.NetworkInformation on Unix 2486 area-System.Net Implement System.Net.WebSockets.Client on Unix 2487 area-System.Net Implement System.Net.Utilities on Unix 2488 area-System.Net GetResponseAsync_ServerNameNotInDns_ThrowsWebException failing locally "I just upgraded to Windows 10 RTM and VS2015 RTM. Not sure if it's related, but this test is now failing for me locally. ``` Discovering: System.Net.Requests.Tests Discovered: System.Net.Requests.Tests Starting: System.Net.Requests.Tests System.Net.Requests.Test.HttpWebRequestTest.GetResponseAsync_ServerNameNotInDns_ThrowsWebException(remoteServer: Uri { AbsolutePath = ""/get"", AbsoluteUri = ""http://httpbin.org/get"", Authority = ""httpbin.org"", DnsSafeHost = ""httpbin.org"", Fragment = """", ... }) [FAIL] Assert.True() Failure Stack Trace: \corefx\src\System.Net.Requests\tests\HttpWebRequestTest.cs(398,0): at System.Net.Requests.Test.HttpWebRequestTest.GetRes ponseAsync_ServerNameNotInDns_ThrowsWebException(Uri remoteServer) System.Net.Requests.Test.HttpWebRequestTest.GetResponseAsync_ServerNameNotInDns_ThrowsWebException(remoteServer: Uri { AbsolutePath = ""/get"", AbsoluteUri = ""https://httpbin.org/get"", Authority = ""httpbin.org"", DnsSafeHost = ""httpbin.org"", Fragment = """", ... }) [FAIL] Assert.True() Failure Stack Trace: \corefx\src\System.Net.Requests\tests\HttpWebRequestTest.cs(398,0): at System.Net.Requests.Test.HttpWebRequestTest.GetRes ponseAsync_ServerNameNotInDns_ThrowsWebException(Uri remoteServer) Finished: System.Net.Requests.Tests ``` " 2490 area-System.Net Open Source System.Net.Websockets.Client 2492 area-System.Net Open source System.Net.Security 2493 area-System.Net Avoid unnecessary byte[] allocations in HttpContent `HttpContent.ReadAsStringAsync()` has a bunch of calls to `Encoding.GetPreamble()` as part of its encoding detection. `GetPreamble()` creates a new `byte[]` each time it is called. These `byte[]` allocations can be avoided. 2496 area-Serialization Fix serialization test failure in Minsk timezone Fix #2287 2499 area-System.Net Replace ad-hoc definitions of SafeHandleZeroOrMinusOneIsInvalid with a single definition. Just as it says. There are at least two or three distinct definitions of this type, all with identical semantics. 2500 area-System.Net Event logging for System.Net.* Event logging for these assemblies is currently unimplemented: we need to design and implement a solution. 2501 area-System.Net Remove dead code - SafeWinHttpHandleWithCallback SafeWinHttpHandleWithCallback is a better pattern; SafeWinHttpHandle is probably a better name. The implementation for the former should probably be folded into the latter. 2502 area-Serialization XmlUTF8TextWriter has not implement WriteStartElementAsync or WriteEndElementAsync WCF's async operations need to call these methods. See https://github.com/dotnet/wcf/issues/221. 2503 area-System.Net Move System.Net.* strings to resources as appropriate.` Just what it says on the tin. 2504 area-Serialization Add FlushAsync and WriteBase64Async in XmlDictionaryWriter WCF needs to use FlushAsync and WriteBase64Async. See https://github.com/dotnet/wcf/issues/222. 2505 area-System.Net Replace single-item pinned buffers in System.Net.WebSockets.Client with PinnableBufferCache. 2506 area-System.Net Optimize locking patterns in System.Net.WebSockets.Client. For example, release the lock between operations and check, under the lock, that the state is still valid before continuing. 2507 area-System.Net Optimize factoring over operations between WinHttpWebSocket and WinHttpWebSocketCallback. It may be profitable to move some code from WinHttpWebSocket to WinHttpWebSocketCallback. 2508 area-System.Net Release pinned buffers at correct time in WinHttpResponseStream, WinHttpWebSocket This is a potential bug: pinned buffers must not be released until it is guaranteed that no further operations will be made that use those buffers. 2509 area-System.Net Handle SSL-related failures in WinHttpWebSocketCallback. This code needs to be shared with WinHttpHandler. 2511 area-System.Net Initial commit of System.Net.WebSockets.Client. 2512 area-System.IO Final major update to the tests for System.IO.FileSystem "- Resolves #2458. FileInfo.Open(FileMode.Append) didn't allow appending per so I modified FileInfo to allow it - Resolves #2460. File.Move and FileInfo.MoveTo no longer always overwrite the destination file. - Resolves #2459. Reenabled Directory_CreateDirectory.DriveLetter_Unix as it's previous failures were not the fault of the method but of the system. - Added Interop.Link to the Interop files for the implementation of File.Move - Consolidated all of the related files that did nothing but inherit from other classes (e.g. File_GetFiles_str and File_GetFiles_str_str were consolidated into File_GetFiles) - Renamed all files to only include parameters in the filename if there are multiple files for a test. - Removed the ""partial"" keyword from some test classes - Finished overhauling all test cases that don't directly require solutions to Build System issues before being re-added (e.g. those that require networked drives or altering the current directory) - Added some tests to increase code coverage slightly in some cases. - All classes and files have been renamed to match the standard set by FileStream: only include parameters in the test/class name if there are overloads - Modified the Unix behavior for distinguishing between a FileNotFoundException and a DirectoryNotFoundException to match that of Windows. " 2522 area-System.Security RSAOpenSsl should support PSS signing RSAOpenSsl::SignHash throws when the signing mode is other than Pkcs1. OpenSSL's RSA_sign function only does PKCS1, and that's how signing currently works. PSS is theoretically available, but needs to be done using lower level API. Remember to keep it bound to OpenSsl 1.0.1 (if possible... if not, it might have to just throw on OSX). 2523 area-System.Security RSAOpenSsl should support OAEP padding using a SHA-2 algorithm Currently it only supports Pkcs1 and OaepSha1. Other hashing algorithms would work, but it requires doing the OAEP manually, since OpenSSL doesn't expose a convenient wrapper function for this. 2524 area-System.Net Reduce HttpWebResponse header allocations Removed unnecessary ToUpperInvariant allocation in WinHttpHandler. HttpWebResponse created a StringBuilder for every header in the response to handle possible multi-values. Common case is a single value so lazy create the buffer when there is more than 1. 2525 area-System.Net System.Uri.GetComponents behavior and MSDN documentation "`System.Uri.GetComponents` method behaves different from the MSDN documentation. https://msdn.microsoft.com/en-us/library/system.uri.getcomponents%28v=vs.110%29.aspx The documentation says: > When `Query` or `Fragment` is specified alone, the return value includes the delimiter. But actually, this method returns value without delimiter. ``` csharp var uri = new Uri(""http://www.example.com/path/to/sample.html?key=value#fragment""); // Expected: ""path/to/sample.html"" (without delimiter; not ""/path/to/sample.html"") Debug.Assert(uri.GetComponents(UriComponents.Path, UriFormat.SafeUnescaped).Equals(""path/to/sample.html"")); // Expected: ""/path/to/sample.html"" (with delimiter) Debug.Assert(uri.GetComponents(UriComponents.Path | UriComponents.KeepDelimiter, UriFormat.SafeUnescaped).Equals(""/path/to/sample.html"")); // FAIL (Expected: ""?key=value"", Actual: ""key=value"") Debug.Assert(uri.GetComponents(UriComponents.Query, UriFormat.SafeUnescaped).Equals(""?key=value"")); // Expected: ""?key=value"" (with delimter) Debug.Assert(uri.GetComponents(UriComponents.Query | UriComponents.KeepDelimiter, UriFormat.SafeUnescaped).Equals(""?key=value"")); // FAIL (Expected: ""#fragment"", Actual: ""fragment"") Debug.Assert(uri.GetComponents(UriComponents.Fragment, UriFormat.SafeUnescaped).Equals(""#fragment"")); // Expected: ""#fragment"" (with delimiter) Debug.Assert(uri.GetComponents(UriComponents.Fragment | UriComponents.KeepDelimiter, UriFormat.SafeUnescaped).Equals(""#fragment"")); ``` I needed `UriComponents.KeepDelimiter` flag just like `UriComponents.Path` to get result with delimiter. I'm not sure that this is a bug (maybe this is a documentation issue; even Mono behaves like this), but I want this ""issue"" to be reviewed anyway. " 2531 area-System.Net Remove StreamToStreamCopy in HttpClient This change removes the internal `StreamToStreamCopy` class, which is only used by `StreamContent.SerializeToStreamAsync`, thus avoiding the instance allocation of the class. `Stream` already has a built-in `CopyToAsync` helper that can be used, that `MemoryStream` [overrides](https://github.com/dotnet/coreclr/blob/ef1e2ab328087c61a6878c1e84f4fc5d710aebce/src/mscorlib/src/System/IO/MemoryStream.cs#L455-L457) to provide a more efficient implementation when the destination is another MemoryStream. Also some minor cleanup: I made some fields `readonly` that could be, and removed an unused using. Question about the existing behavior: If an exception is thrown during the copy operation, the source stream won't be disposed (if it can be disposed). Is this is by design? I was going to wrap the `await _content.CopyToAsync(...)` in a try/finally with the dispose in the finally, but I didn't want to change the existing behavior if it is by design. 2532 area-System.Net Cleanup ByteArrayContent - Use `Task.FromResult` instead of `TaskCompletionSource` (avoids the allocation) - Make readonly fields `readonly` - Remove unused `using` 2534 area-System.Net Make StreamToStreamCopy static Avoids the unnecessary instance allocation in `StreamContent.SerializeToStreamAsync()`. 2535 area-System.Text System.Text.Encoding HeaderName missing Was this property removed on purpose? Or is it just not yet implemented? If it _has_ been purposely removed, what should I be using instead if I want the name to use in email headers? 2537 area-System.IO Win32FileSystem File.Move is inconsistent with its Exception throws "There is an inconsistent behavior in File.Move with regards to non existent directories in paths that is present in both the desktop and core frameworks. If the parent directory of the source file is nonexistent, a FileNotFoundException is thrown. If the parent directory of the dest file is nonexistent, a DirectoryNotFoundException is thrown. Example: ``` File.Create(""test1"").Dispose(); File.Move(Path.Combine(Directory.GetCurrentDirectory(), ""test2"", ""test3""), ""test4""); // throws a FileNotFoundException File.Move(""test1"", Path.Combine(Directory.GetCurrentDirectory(), ""test2"", ""test3"")); // throws a DirectoryNotFoundException ``` The reason for this is that File.Move makes a call to ""InternalExists"" to check that the sourceFile exists in full and if it doesn't then a FileNotFoundException is thrown regardless of which part of the path caused the failure. Removing this check is simple and will leave path checking to the FileSystem implementation (which would be preferable), but it will need to be mirrored in the desktop as well or ignored/worked around on both. " 2538 area-System.Globalization TimeZoneInfo should have consistent Ids across Windows and Linux "Currently Windows has TimeZoneInfo.Id values like ""Pacific Standard Time"" and ""Eastern Standard Time"". But on Linux they are ""America/Los_Angeles"" and ""America/New_York"". We should make these Ids consistent across platforms so the same code can run on both Windows and Linux. Currently if you call TimeZoneInfo.FindSystemTimeZoneById you need to pass in different strings between Windows and Linux. " 2542 area-System.Net Unify System.Net.WebSockets.Client's WinHttpException with System.Net.Http's WinHttpException. 2546 area-System.Security Many deprecation warnings in System.Security.Cryptography.Native on Mac OS X I'm about to unblock the mac build by disabling the warnings, but building System.Security.Crytpography.Native on Mac OS X produces many deprecation warnings. 2551 area-System.Reflection BugFix: Signature Decoder not providing ValueType or Class flag This change would fix #2314. I just added to flags into the provider interface so that the api consumers can retrieve the information in case they need it. In order to do these they need to set the flag `bool NeedProjectedFlags` to `true` . In this case they could know if it is a valuetype or class through the `SignatureTypeCode TypeCode` property added in `ISignatureTypeProvider` as well. Adding @AlexGhiondea 2553 area-System.Net Fix regression in HttpMethod.GetHashCode() A PR from last month changed how GetHashCode() works. This ended up breaking tests against the .NET Native build of System.Net.Http which shares the same source code as CoreFx. So, reverting the change in GetHashCode() and porting additional ToF tests to GitHub. 2554 area-System.Net Update README.md for System.Net.WebSockets.Client System.Net.WebSockets.Client source code was recently ported to GitHub. Updating the README.md for that. cc: @CIPop @SidharthNabar @pgavlin @stephentoub @ericstj 2555 area-System.Net Address feedback on xplat implementation of HttpClient 2556 area-System.Net Consider changing HttpMethod to do equality comparisons case-sensitive Reference PR #2553. Consider changing the `System.Net.Http.HttpMethod` class so that equality comparison between two `HttpMethod` objects is done in a case-sensitive manner (aligning with the latest HTTP RFC) instead of the current case-insensitive manner. 2557 area-System.Net HttpClient via libcurl : proxy support This commit introduces integrates CURL handler with the proxy functionality offered by libcurl 2558 area-System.IO Modified calls to GetTestFilePath within loops to not use the same filename - Added an overload to GetTestFilePath that takes an integer index parameter to add to the generated filename so that calls to the function within loops will be able to differentiate themselves - Resolves #1434 2559 area-System.Net Initial commit of System.Net.Primitives. 2561 area-System.Net Optimize HttpMethod.GetHashCode() Avoid unnecessary allocations associated with the use of `ToUpperInvariant()`, and test improvements. Also, some minor cleanup (made `_method` `readonly`, removed unused usings, and removed trailing whitespace). 2575 area-System.Data SQL server alias recognized with CLR runtime but not CoreCLR "I wrote a test console application and set up a SQL configuration alias in SQL server configuration manager. When running my application with the dnx 4.51 runtime, the database connection is established with no problem and the application functions normally If I switch the runtime to dnx core 5.0, any attempts to connect to the database from the application fail with the SqlException 'The network path was not found.' Replacing the alias in the connection string with the actual server name resolves the issue. The connection string I'm using is included below: ``` ""connectionString"": ""Server=aliasName;Database=dbName;Trusted_Connection=True;MultipleActiveResultSets=true"" ``` Perhaps the System.Data.SqlClient coreCLR library does not support SQL aliases and this is not a bug. I just think the different behavior between runtimes is a bit odd and the error it produces is not helpful in deciphering the problem. " 2576 area-System.Threading System.Threading.Thread should support all runtimes The current System.Threading.Thread (and System.Threading.ThreadPool) packages only support desktop and CoreCLR. The problem is that with a name like System.Threading.\* and with threads being a core part of many apps, people won't know (or care!) that UWP apps will lose out on that library because only Task is supported. The best solution is to implement Thread/ThreadPool for UWP so that people can use the threading model they're most comfortable with and what's most appropriate for the job. Forcing Task for UWP is counterproductive if it'll mean that UWP will lose out on many libraries that choose to use Thread anyway. When it comes to creation/consumption of .NET Core packages, right now ASPNet is the biggest driver. As much as we want people to use UWP, it's just smaller for now. That puts even greater pressure on UWP as library authors will either not know that Thread isn't available on UWP or care even if they do. .NET Core is supposed to be unifying the BCL and shielding libraries from platform/runtime differences. Please don't fragment it with something as central as Thread. 2582 area-Infrastructure Build warnings from outer-loop only test projects We have a bunch of these on every build: ``` EXEC : warning : System.Xml.RW.FactoryReader.Tests has no tests to run ``` My understanding is that these libraries contain only outer loop tests by design. in that case, we should adjust things to not warn in that case. I also wonder if we really need so many different XML test assemblies... 2583 area-System.Security PfxTests.TestPrivateKey intermittent failure I can't repro it, but I got the following once: ``` System.Security.Cryptography.X509Certificates.Tests.PfxTests.TestPrivateKey [FAIL] System.Security.Cryptography.CryptographicException : OpenCSP failed with error code 2148073494. Stack Trace: D:\Src\corefx\src\System.Security.Cryptography.Csp\src\System\Security\Cryptography\CapiHelper.cs(306,0): at Internal.NativeCrypto.CapiHelper.CreateProvHandle(CspParameters parameters, Boolean randomKeyContainer) D:\Src\corefx\src\System.Security.Cryptography.Csp\src\System\Security\Cryptography\CapiHelper.cs(698,0): at Internal.NativeCrypto.CapiHelper.GetKeyPairHelper(CspAlgorithmType keyType, CspParameters parameters, Boolean randomKeyContainer, Int32 keySize, SafeProvHandle& safeProvHandle, SafeKeyHandle& safeKeyHandle) D:\Src\corefx\src\System.Security.Cryptography.Csp\src\System\Security\Cryptography\RSACryptoServiceProvider.cs(89,0): at System.Security.Cryptography.RSACryptoServiceProvider.GetKeyPair() D:\Src\corefx\src\System.Security.Cryptography.Csp\src\System\Security\Cryptography\RSACryptoServiceProvider.cs(69,0): at System.Security.Cryptography.RSACryptoServiceProvider..ctor(Int32 keySize, CspParameters parameters, Boolean useDefaultKeySize) D:\Src\corefx\src\System.Security.Cryptography.Csp\src\System\Security\Cryptography\RSACryptoServiceProvider.cs(50,0): at System.Security.Cryptography.RSACryptoServiceProvider..ctor(CspParameters parameters) D:\Src\corefx\src\System.Security.Cryptography.X509Certificates\src\Internal\Cryptography\Pal.Windows\CertificatePal.PrivateKey.cs(49,0): at Internal.Cryptography.Pal.CertificatePal.get_PrivateKey() D:\Src\corefx\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\RSACertificateExtensions.cs(55,0): at System.Security.Cryptography.X509Certificates.RSACertificateExtensions.GetRSAPrivateKey(X509Certificate2 certificate) D:\Src\corefx\src\System.Security.Cryptography.X509Certificates\tests\PfxTests.cs(70,0): at System.Security.Cryptography.X509Certificates.Tests.PfxTests.TestPrivateKey() Finished: System.Security.Cryptography.X509Certificates.Tests === TEST EXECUTION SUMMARY === System.Security.Cryptography.X509Certificates.Tests Total: 157, Errors: 0, Failed: 1, Skipped: 0, Time: 0.854s ``` 2585 area-System.Reflection Merge master to dev/metadata 2586 area-System.Reflection SignatureDecoder isValueType not set correctly for some projected types and should be optional Follow-up to PR #2551 a. isValueType is not set correctly when a type is projected from value type to class or vice versa. b. There should be an opt-in option to pas non-null isValueType. By default, it should always be null and when option is on, it should compute isValueType (including possible projection case) cc @nguerrera 2588 area-System.IO Modified the Unix implementation of FileMove and fixed the corresponding broken FileSystemWatcher tests - Resolves #2460 - Changed the way the UnixFileSystem does File moving; it was previously implemented using the rename syscall but now uses a combination of link and unlink. The side effect of this change is that the FileSystemWatcher will now treat File.Move as a deletion. - Removed all tests from FileSystemWatcher that made calls to File.Move and consolidated them into a new file: FileSystemWatcher.MoveFile.cs. This new file separates tests into Windows and Unix categories and uses Theories to supply the input WatcherChangeType and whether a move should trigger an event for the given type. If we later decide to change the Unix behavior of MoveFile, changing the FileSystemWatcher tests will be a simple matter of flipping the boolean values for the affected tests. 2589 area-System.Net Update README.md for System.Net.Primitives System.Net.Primitives source code is now on GitHub. Updating the README.md. 2591 area-System.Net Cleanup CookieCollection - Public surface area now matches the [reference assembly contract](https://github.com/dotnet/corefx/blob/b84605422a74d69e7aa61a25d8760bfe2fa32296/src/System.Net.Primitives/ref/System.Net.Primitives.cs#L47-L55) and dead code removed - Removed the `CookieCollectionEnumerator` (`_list.GetEnumerator()` used instead). This removes the need for the `_version` field as `List` already tracks this - Avoid the enumerator allocation in `Add(CookieCollection)` by enumerating `_list` - Removed unused internal constructor and associated field - Made the `_list` field readonly - Removed unused usings - Removed trailing whitespace Fixes #2593 2593 area-System.Net Remove dead code related to CookieCollection.IsReadOnly from System.Net.Primitives **Edit:** `IsReadOnly` is dead code that should be removed from CoreCLR. --- Original issue description (kept here so the comments below make sense): > The [default constructor](https://github.com/dotnet/corefx/blob/63ff03fce5ab7c7ac6920d99af148a447d04c822/src/System.Net.Primitives/src/System/Net/CookieCollection.cs#L33) sets this to `true`, but `CookieCollection` is always mutable (same as on the desktop framework) -- there's nothing in its implementation that enforces it being read-only. > > Is it too late to remove this public property from the System.Net.Primitives contract? (I'm assuming it can't be removed, but figured I'd ask anyway). > > If it can't be removed, should the default value be changed to `false` to reflect the actual behavior? However, this would go against the [MSDN documentation](https://msdn.microsoft.com/en-us/library/w6x28886%28v=vs.110%29.aspx) for the desktop framework, which states the default value is `true`. 2595 area-System.Net Implement System.Net.Primitives for *nix. Platform abstractions have been added for IP address parsing and host information. The closest APIs that most POSIX systems have to the NTDLL IP address parsing APIs are getaddrinfo/getnameinfo. In the case of IPv6 in particular, inet_ntop/inet_pton are not sufficient, as these APIs do not support scope IDs. 2597 area-System.Net HttpClient xplat: Basic request and response processing - Also addressed some of the comments in https://github.com/dotnet/corefx/issues/2555 2598 area-System.Net Fix HTTPS checks in xplat HttpClient cc: @davidsh @stephentoub @CIPop @SidharthNabar @pgavlin 2604 area-Infrastructure run-test.sh deploys WP8 dataflow if present, fails ETW test On my machine (might have different behavior based on file system order), running with System.Threading.Tasks.Dataflow and System.Threading.Tasks.Dataflow.WP8 dlls in the corefx bin path, the test overlay gets the WP8 build, which doesn't have FEATURE_TRACING and the ETW test fails 2608 area-System.Net From Feedback in issue 2149, Propagate CancellationToken in calls to TrySetCanceled: As a minor improvement, using the new TrySetCanceled overload, you could make this state.Tcs.TrySetCanceled(state.CancellationToken) ... that just helps to ensure that the token is propagated to anyone getting an OperationCanceledException later on. 2610 area-System.IO FileSystem GetLastWriteTime and GetLastAccessTime don't properly convert values to Local DateTimeKind "The following test has different results for each platform: ``` FileInfo testFile = new FileInfo(""test.txt""); testFile.Create().Dispose(); DateTime dt = new DateTime(2014, 12, 1, 12, 0, 0, DateTimeKind.Local); Console.WriteLine(""regular: "" + dt.ToString()); Console.WriteLine(""local: "" + dt.ToLocalTime().ToString()); Console.WriteLine(""utc: "" + dt.ToUniversalTime().ToString()); File.SetLastWriteTime(testFile.FullName, dt); var res = File.GetLastWriteTime(testFile.FullName); Console.WriteLine(""result: "" + res.ToString()); Console.WriteLine(""result.toLocal(): "" + res.ToLocalTime().ToString()); Console.WriteLine(""result.ToUTC(): "" + res.ToUniversalTime().ToString()); Console.WriteLine(); ``` On Desktop Framework: ``` regular: 12/1/2014 12:00:00 PM local: 12/1/2014 12:00:00 PM utc: 12/1/2014 8:00:00 PM result: 12/1/2014 12:00:00 PM result.toLocal(): 12/1/2014 12:00:00 PM result.ToUTC(): 12/1/2014 8:00:00 PM ``` On Win32 Core: ``` regular: 12/1/2014 12:00:00 PM local: 12/1/2014 12:00:00 PM utc: 12/1/2014 8:00:00 PM result: 12/1/2014 12:00:00 PM result.toLocal(): 12/1/2014 4:00:00 AM result.ToUTC(): 12/1/2014 8:00:00 PM ``` On Linux Core: ``` regular: 12/01/2014 12:00:00 local: 12/01/2014 12:00:00 utc: 12/01/2014 4:00:00 result: 12/01/2014 04:00:00 result.toLocal(): 12/01/2014 12:00:00 result.ToUTC(): 12/01/2014 20:00:00 ``` The issue lies in the conversions from DateTimeOffsets to DateTimes: nowhere is DateTimeKind (i.e. local or UTC) considered when reading the attributes of a file. To make the behavior in line with the Desktop, we should convert all filesystem times (CreationTime, WriteTime, AccessTime) to DateTimeKind.Local when converting them from a DateTimeOffset to a DateTime. Thanks to @eerhardt for help finding the lack of conversion. " 2611 area-System.IO Fixed AccessTime, WriteTime, and CreationTime to return a DateTime object with properly set DateTimeKind - The implementations of GetLastAccessTime, GetCreationTime, and GetLastWriteTime were not properly converting the read values to LocalTime for either Windows or Unix. - Resolves #2603, #2610 2613 area-System.Diagnostics ProcessThreadTests.TestStartTimeProperty failed on Linux in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst_prtest/1488/console ``` 20:32:15 System.Diagnostics.ProcessTests.ProcessThreadTests.TestStartTimeProperty [FAIL] 20:32:15 Assert+WrapperXunitException : File path: d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Diagnostics.Process\tests\System.Diagnostics.Process.Tests\ProcessThreadTests.cs. Line: 85 20:32:15 ---- Assert.InRange() Failure 20:32:15 Range: (635742559171500970 - 635742559171722530) 20:32:15 Actual: 635742559171972800 20:32:15 Stack Trace: 20:32:16 at Assert.WrapException(Exception e, String callerFilePath, Int32 callerLineNumber) 20:32:16 at Assert.InRange[T](T actual, T low, T high, String path, Int32 line) 20:32:16 at System.Diagnostics.ProcessTests.ProcessThreadTests.TestStartTimeProperty() 20:32:16 ----- Inner Stack Trace ----- 20:32:16 at Assert.InRange[T](T actual, T low, T high, String path, Int32 line) 20:32:20 Finished: System.Diagnostics.Process.Tests 20:32:20 20:32:20 === TEST EXECUTION SUMMARY === 20:32:20 System.Diagnostics.Process.Tests Total: 55, Errors: 0, Failed: 1, Skipped: 0, Time: 6.307s ``` 2618 area-System.Runtime TimeZoneInfo.TransitionTime cannot support Israel, et al. time zones on Linux "It currently isn't possible to support Israel, Asia/Amman, Asia/Gaza, Asia/Hebron, Pacific/Fiji and America/Godthab daylight savings rules in the POSIX string that is used on Linux to describe the daylight savings transitions for years after the last transition stored in the file. Here is Israel's POSIX string: IST-2IDT,M3.4.4/**26**,M10.5.0 This is saying that daylight savings starts 26 hours after midnight of the 4th Thursday of March. ""On July 8, 2013, the Israeli Knesset approved the bill to extend IDT even further. According to the bill, IDT will begin on the Friday before the last Sunday of March, and end on the last Sunday of October."" From https://en.wikipedia.org/wiki/Israel_Summer_Time So this might not be the last Friday of the month. Some years it is the last Friday of the month, some years it is the 2nd to last. This rule isn't possible to describe in the current [TimeZoneInfo.TransitionTime structure](https://msdn.microsoft.com/en-us/library/vstudio/system.timezoneinfo.transitiontime%28v=vs.110%29.aspx). It can only specify the 4th Friday, or the last Friday, there is no way to say ""the Friday before the last Sunday"". To fix this, we should add a ""TimeSpan Offset"" to TransitionTime that we can use to convert the string ""M3.4.4/26"" into a TransitionTime structure. Here the TimeSpan Offset = 26 hours, and month = 3, week = 4, day = 4. The ""TimeOfDay"" property would be ignored in this case. Note that America/Godthab uses a negative time offset: WGT3WGST,M3.5.0/-2,M10.5.0/-1. " 2619 area-System.Reflection Three TypeExtensions tests failing on Linux with xUnit 2.1.0 A few of the reflection tests seem to be failing on Linux when run with xUnit 2.1.0. These tests are doing fairly simple stuff, just subscribing some events and then triggering them, and checking the state afterwards. ``` System.Reflection.Compatibility.UnitTests.EventInfoAddEventHandler.PosTest2 System.Reflection.Compatibility.UnitTests.EventInfoAddEventHandler.PosTest3 System.Reflection.Compatibility.UnitTests.EventInfoAddEventHandler.PosTest1 ``` 2621 area-System.Net Fix casing to unblock Linux native builds I had hoped that this was going to be fixed by PR #2591, but that PR appears to be on hold for the moment. This change fixes the casing in the proj file so that native Linux builds will pass. 2624 area-System.Reflection Bump file version of S.R.Metadata to match assembly version 2632 area-System.IO DirectoryInfo.ToString inconsistent with Desktop On Desktop creating a DirectoryInfo with a full path returns a full path. Creating a DirectoryInfo with a short path, or creating it from a parent (eg: CreateSubdirectory, EnumerateDirectories,etc) returns just the directory name. On CoreCLR we are always trimming to just the directory name. I traced this behavior back to an IFDEF in the original code https://github.com/dotnet/coreclr/blob/master/src/mscorlib/src/System/IO/DirectoryInfo.cs#L639-L643. It seems that the original CoreCLR implementation (Silverlight/phone) hardcoded the short name. We should prefer the desktop behavior instead of the current. The current behavior is due to the legacy behavior of SL trying to sandbox the app. This is not a goal for CoreCLR and we don't have a back-compat burden for the current implementation. We also need to examine similar cases in other FileSystem types. 2635 area-System.Security Ensure that X509Certificates file tests run the gamut of supported formats. Updating the testdata package isn't the most friendly of things to do right now, so tracking this in an issue for later. - [X] X509-DER - [ ] X509-PEM - [X] PKCS12 - [ ] PKCS7 - [ ] PKCS7-PEM 2640 area-System.IO [IO.Pipes] Outer cancellation of WaitForConnectionAsync/ReadAsync/WriteAsync "Consider the following code ``` c# [DllImport(""api-ms-win-core-io-l1-1-0.dll"", SetLastError = true)] internal static unsafe extern bool CancelIoEx(SafeHandle handle, NativeOverlapped* lpOverlapped); public static async Task CanBeCanceled_False() { using (NamedPipeServerStream server = new NamedPipeServerStream(""unique3"", PipeDirection.InOut, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous, 0, 0)) { Task waitForConnectionTask = server.WaitForConnectionAsync(CancellationToken.None); // cannot be canceled unsafe { CancelIoEx(server.SafePipeHandle, null); } await waitForConnectionTask; // throws OperationCanceledException } } public static async Task CanBeCanceled_True() { using (NamedPipeServerStream server = new NamedPipeServerStream(""unique3"", PipeDirection.InOut, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous, 0, 0)) { var ctx = new CancellationTokenSource(); Task waitForConnectionTask = server.WaitForConnectionAsync(ctx.Token); // could be canceled unsafe { CancelIoEx(server.SafePipeHandle, null); } await waitForConnectionTask; // throws IOException: IO operation was aborted unexpectedly. } } ``` Is it intentional behavior or it's a bug that we receive `OperationCanceledException` even if we couldn't cancel in a normal way (without interop) ? " 2648 area-System.ComponentModel RangeAttribute with string constructor fails on various locales "`RangeAttribute(Type, string, string)` can be used to validate various types implementing `IComparable`. However, it does the conversion from the string parameters using the current culture, which is wrong. Simple repro: ``` Thread.CurrentThread.CurrentCulture = new CultureInfo(""fr""); new RangeAttribute(typeof(decimal), ""0.1"", ""0.9"").IsValid(123m) ``` fails with a `FormatException`: _0.1 is not a valid value for Decimal._ While in this sample the strings could be adjusted at runtime to represent numbers using the current culture, the typical usage of an attribute is ``` [Range(typeof(decimal), ""0.1"", ""0.9"")] public decimal Value { get; set; } ``` where the values **must** be constants, and can't change depending on the culture. Culprit line: https://github.com/dotnet/corefx/blob/master/src/System.ComponentModel.Annotations/src/System/ComponentModel/DataAnnotations/RangeAttribute.cs#L203 The fix can't be to simply change `CurrentCulture` to `InvariantCulture` here, since this will alter the runtime validation behavior: `InvariantCulture` should be used to convert the `min` and `max` parameters (maybe falling back to `CurrentCulture` for back-compat, if the constructor was used directly), but `CurrentCulture` should be kept to validate the user-entered value. If accepted, I can add some unit tests and a fix for this issue. " 2651 area-System.Net Automatic Content Decompression Modified CurlHandler to set the curl options for automatic response content decompression. 2658 area-Serialization Fix one of DCS constructors that cause NRE in wcf Verified that this fixes #2435 by aligning with the behavior on Desktop cc: @mconnew 2666 area-System.Reflection Bug fix: When decoding a modified type it was throwing a BadImageForm… …atException, the previous change to get the valuetype and class prefix flag broke that because didn't update DecodeModifiedType to read a compressed integer instead of reading a SignatureTypeCode. 2667 area-System.Security TestLoadSignedFile failure in CI See: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/2796/testReport/junit/System.Security.Cryptography.X509Certificates.Tests/LoadFromFileTests/TestLoadSignedFile/ ``` MESSAGE: System.Security.Cryptography.CryptographicException : Error occurred during a cryptographic operation. +++++++++++++++++++ STACK TRACE: at Internal.Cryptography.Pal.CertificatePal.GetSignerInPKCS7Store(SafeCertStoreHandle hCertStore, SafeCryptMsgHandle hCryptMsg) in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\Internal\Cryptography\Pal.Windows\CertificatePal.Import.cs:line 142 at Internal.Cryptography.Pal.CertificatePal.FromBlobOrFile(Byte[] rawData, String fileName, String password, X509KeyStorageFlags keyStorageFlags) in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\Internal\Cryptography\Pal.Windows\CertificatePal.Import.cs:line 84 at Internal.Cryptography.Pal.CertificatePal.FromFile(String fileName, String password, X509KeyStorageFlags keyStorageFlags) in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\Internal\Cryptography\Pal.Windows\CertificatePal.Import.cs:line 29 at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(String fileName, String password, X509KeyStorageFlags keyStorageFlags) in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\X509Certificate.cs:line 72 at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(String fileName) in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\X509Certificate.cs:line 55 at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(String fileName) in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\X509Certificate2.cs:line 48 at System.Security.Cryptography.X509Certificates.Tests.LoadFromFileTests.TestLoadSignedFile() in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\tests\LoadFromFileTests.cs:line 127 ``` 2676 area-System.Net Merge changes from TFS 2684 area-System.IO FSW tests don't verify path information The tests are only checking that events are occurring, but not that they're for the right/expected paths. 2688 area-System.Reflection System.Reflection.Metadata needs a MarshalDecoder for marshalling descriptor types. It needs a marshal decoder such as CustomAttributes decoder and SignatureDecoder to decode marshalling descriptor since it specifications on ECMA spec 355 are diferent. cc @nguerrera @AlexGhiondea 2699 area-System.IO Code cleanup for System.IO.Packaging "- [x] ContentType.cs - line 234 - use String.Equals instead of String.Compare - [x] ContentType.cs - 83 - if (contentType.Length == 0) - [x] PackagePart.cs - 849 - return !s.CanRead && !s.CanSeek && !s.CanWrite; - [ ] Package.cs - 976 - What's the purpose of the try block with an empty finally? - [x] PackageUriHelper.cs - 1297 - at least one other place where the same thing was declared. Should they be consolidated? - [x] PackageUriHelper.cs - 1025 - Rather than using comments, these could use named arguments. - [x] PackageUriHelper.cs - 991 - There are various places thus far in the implementation that use '/' hardcoded... should they instead be using ForwardSlashChar? - [ ] PackageUriHelper.cs - 926 - Why not just: return the_condition; - [x] PackUriHelper.cs - 695 - the chars array could be stackalloc'd instead of heap allocated. - [x] PackUriHelper.cs - 451 - Any reason not to combine these Path.Combine calls? - [x] InternalRelationshipCollection.cs - 592 - Debug.Fail? - [ ] InternalRelationshipCollection.cs - 325 - Formatting off - [ ] IgnoreFlushAndCloseStream.cs - 18 - This comment appears out-of-date: there is no IgnoreFlushAndClose property, and Flush/Dispose never delegate to the wrapped stream's corresponding members. - [ ] IgnoreFlushAndCloseStream.cs - 48 - get { return !_disposed && _stream.CanRead; } - [ ] FileFormatException.cs - 139 - readonly? - [x] ContentType.cs - 666 - This loop could be replaced by a call to Array.IndexOf. - [x] ContentType.cs - 620 - return IsAsciiLetter(character) || (character >= '0' && character <= '9'); - [x] ContentType.cs - 561 - It seems a bit odd to be explicit about the kind of comparison we want for the characters and to be doing a Compare rather than Equal when what we're checking for is just an empty string. - [x] ContentType.cs - 527 - if (string.IsNullOrEmpty(token)) - [x] ContentType.cs - 531 - The body of this loop could be simplified to just: if (!IsAsciiLetterOrDigit(token[i]) && !IsAllowedCharacter(token[i])) throw new ArgumentException(SR.InvalidToken); - [ ] ContentType.cs - 398 - There's a lot of string manipulation / allocation / etc. happening in these various helper functions. Doesn't need to be fixed now, as it's not clear that it's a problem. But it'd be interesting to do some perf testing / profiling and see what pops as hot spots that we'll potentially want to fix. - [x] ContentType.cs - 390 - ""ans"" => ""and"" - [ ] ContentType.cs - 225 - Do we know that MoveNext is always going to return true here such that Current will be valid? - [ ] ContentType.cs - 89 - It doesn't appear that we know at this point that contentType isn't empty. Couldn't this result in indexing out of bounds into contentType? - [x] ContentType.cs - 262 - ""paramterKey"" => ""parameterKey"" - [ ] PackUriHelper.cs - 791 - Could this just do a ordinal-ignorecase comparison rather than allocating new strings with ToUpperInvariant? - [x] ContentType.cs - 323 - Is this type used anywhere? It appears to be dead code. - [x] ContentType.cs - 294 - Is this type used anywhere? It appears to be dead code. - [x] PackUriHelper.cs - 737 - I don't know if allocations matter here, but you could trivially avoid this allocation by using IndexOf instead of Contains. - [x] ContentType.cs - 604 - Couldn't it use Array.IndexOf? " 2703 area-System.Threading Add a way to wait for a task without throwing `Task.Wait` throws an exception if the task is faulted or cancelled. Sometimes it is necessary to wait for completion but not be interrupted by an exception. Catching and swallowing the exception is a really bad workaround. Using exceptions for control flow is not a good thing. Also, exceptions are extremely slow. 1. The `Wait` method should have an overload that allows the caller to specify that he does not want an exception. 2. There should be a method `Task WhenCompleted(this Task task)` that returns a new task that becomes completed when the base task becomes completed. Cancellation and exceptions should not be propagated. That way we can write `await myTask.WhenCompleted()` to avoid exceptions. 3. In the same spirit as (2) there probably should be `WhenCompletedSuccessfully`, `WhenCancelled` and `WhenFaulted`. These tasks complete when the underlying task transitions to the respective state. They do not ever complete if the underlying task transitions to a different final state. They should _not_ become cancelled in that case because the idea of these methods is that you can avoid exceptions for control flow in a convenient manner. 2705 area-System.Net Fixing multi response header issue along with putting checks in http message parser. …message parser ``` modified: System/Net/Http/Unix/CurlCallbacks.cs ``` 2711 area-System.Reflection Merge master into dev/metadata @tmat @stephentoub @nguerrera 2715 area-System.Security System.Security.Cryptography.Hashing.Algorithms contract reference is forwarding Hi, It seems that in System.Security.Cryptography.Hashing.Algorithms v4.0.0-beta-23127 all types are being forwarded, even though they are implemented in corefx. Is there any reason for this change or is it a mistake? Thanks. 2717 area-System.Threading Four failures in new System.Threading.Timer tests in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_release/946/console ``` TimerFiringTests.Timer_CanDisposeSelfInCallback [FAIL] Assert.True() Failure Expected: True Actual: False Stack Trace: d:\j\workspace\dotnet_corefx_windows_release\src\System.Threading.Timer\tests\TimerFiringTests.cs(126,0): at TimerFiringTests.Timer_CanDisposeSelfInCallback() TimerFiringTests.NonRepeatingTimer_ThatHasAlreadyFired_CanChangeAndFireAgain [FAIL] Should have received a second timer event after changing it Expected: True Actual: False Stack Trace: d:\j\workspace\dotnet_corefx_windows_release\src\System.Threading.Timer\tests\TimerFiringTests.cs(150,0): at TimerFiringTests.NonRepeatingTimer_ThatHasAlreadyFired_CanChangeAndFireAgain() TimerFiringTests.Running_Timer_CanBeFinalizedAndStopsFiring [FAIL] Failed to get first timer fire Expected: True Actual: False Stack Trace: d:\j\workspace\dotnet_corefx_windows_release\src\System.Threading.Timer\tests\TimerFiringTests.cs(160,0): at TimerFiringTests.Running_Timer_CanBeFinalizedAndStopsFiring() TimerFiringTests.MultpleTimers_PeriodicTimerIsntBlockedByABlockedTimer [FAIL] Blocking callback prevented a second timer from firing Expected: True Actual: False Stack Trace: d:\j\workspace\dotnet_corefx_windows_release\src\System.Threading.Timer\tests\TimerFiringTests.cs(184,0): at TimerFiringTests.MultpleTimers_PeriodicTimerIsntBlockedByABlockedTimer() Unhandled Exception: Xunit.Sdk.TrueException: Should have received a second callback while blocked Expected: True Actual: False at Xunit.Assert.True(Nullable`1 condition, String userMessage) at TimerFiringTests.<>c__DisplayClass22.b__20(Object o) at System.Threading.TimerQueueTimer.CallCallbackInContext(Object state) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.TimerQueueTimer.CallCallback() at System.Threading.TimerQueueTimer.Fire() at System.Threading.TimerQueue.FireNextTimers() at System.Threading.TimerQueue.AppDomainTimerCallback() Finished: System.Threading.Timer.Tests ``` 2720 area-System.Net HttpClient xplat: Optimize number of socket callbacks In earlier version, a curl_multi_socket_action was performed everytime the callback was invoked irrespective of whether there was activity on the socket. This in turn causes more callbacks when libcurl tries to read the socket but finds no data. This can be made more efficient by waiting for activity on the socket and informing libcurl accordingly - Also fixed a bug in the timer callback for <=0 timeout values - Fixed a bug in CheckForCompletedTransfers where a mismatch could occur between the easy handle specified in the DONE msg - Fixing a SEGV that can happen due to stack overflow caused by callback getting invoked recursively because curl_multi_socket_action was being called in the callback code cc: @stephentoub @davidsh @CIPop @SidharthNabar @pgavlin 2722 area-System.IO Remove USE_OVERLAPPED from FileSystem We now have a overlapped implementation that works on UWP so we should remove the USE_OVERLAPPED ifdefs. 2727 area-System.Collections IsPrime function unnecessary, GetPrime function inefficient. In System.Collections.HashHelpers and System.Collections.NonGeneric\src\System\Collections\Hashtable and System.Private.DataContractSerialization\src\System\Runtime\Serialization\ObjectToIdCache.cs there are copies of the same function: private static bool IsPrime(int candidate) It is used by: private static int GetPrime(int min) Which is called by: Initialize, ExpandPrime, and TrimExcess They all use a table in GetPrime: E.g. int[] primes = {3, 7, 17, 37, 89, 197, 431, 919, 1931,...7199369}. Notice that each successive prime is larger than twice the preceding one. This appears to be because hashtables are doubled in size when they need to be expanded. But they are only allowed to expand up to 2G. The GetPrime function calls IsPrime to find a prime when the search goes above the limit of the primes table. The ExpandPrime function is the most interesting as it more than doubles the size each time: public static int ExpandPrime(int oldSize) { int newSize = 2 \* oldSize; .... return GetPrime(newSize); Therefore it appears that we don't need the IsPrime function at all. Instead we simply need to expand the primes table up to 2146435069 (largest prime below 2G). {..., 7199369, 21598111, 64794337, 194383039, 583149179, 1749447569, 953375459, 2146435069}; Recommendation: 1) Use a common GetPrime(int min) function instead of having three copies 2) Expand the primes table as described above. 3) Remove the IsPrime function, modify GetPrime to only use the table. 2729 area-System.Numerics Fix very sporadically failing Complex test "I'd previously ""fixed"" this same test by incorporating infinity support, but I needed to do the comparisons based on IsNegative/PositiveInfinity rather than by comparing the values directly. cc: @mellinoe " 2730 area-System.IO Reworked the tests for System.IO.Pipes I made a number of changes to our current Pipes tests and added several hundred new tests for edge cases that were previously ignored, particularly with Named Pipes defined with an InOut PipeDirection - Refactored the great Throws tests currently in place to allow testing pipes with all in/out direction combinations for both Named pipes as well as Anonymous pipes - Replaced all relevant hardcoded pipe names with unique values - Reorganized the tests to use more inheritance and share more code for easy extensibility - All test classes are now in the same namespace: System.IO.Pipes.Tests - Expanded tests for InOut NamedPipes and made all existing In/Out tests also test all valid InOut combinations. - Split the Named and Anonymous tests into two separate folders: NamedPipeTest and AnonymousPipeTest - Increased code coverage across all test classes 2732 area-System.Numerics what set's Vector.IsHardwareAccelerated to true in System.Numerics.Vectors? I am writing a test app, 64 bit using System.Numerics.Vectors 4.1.0. I am trying to understand how `Vector.IsHardwareAccelerated` gets set to true. I’ve tried running this on three Haswell based systems. I get `Vector.IsHardwareAccelerated` as false during debugging. But when I run it from command prompt, I get true in one system and false in other two system. 2733 area-System.IO Cleaned up tests for closed issues Design review feedback brought about the closed issues #2537, #2403, #2402, #2369, #1728. This Commit cleaned up the tests around those decisions 2737 area-System.IO Removed USE_OVERLAPPED from FileSystem All ifdefs around USE_OVERLAPPED have been removed such that the new behavior mimics the old behavior when USE_OVERLAPPED = true. Resolves #2722 2738 area-System.Reflection BlobReader.ReadBoolean: Interpret any non-zero byte as True @nguerrera @davkean 2743 area-System.Security PlatformNotSupportedException on Unix when exporting multiple private keys in a single PFX The beta7 PFX/PKCS12 export code will use OpenSSL's PKCS12_create function, which only supports one private key. OpenSSL doesn't have an inherent limitation against PKCS12 files containing multiple private keys, but the helper function can't handle it; and we're running out of time in beta7 to write it out longhand. 2745 area-System.Security Not all private keys are available when importing a PFX with multiple private keys on Unix The OpenSSL helper function `PKCS12_parse` only emits a single private key (and its associated certificate), the rest of the certificates appear, but as public-key only. Similar to #2743, the fix would be to not use the helper function, but to get into the nitty-gritty with PKCS12 primitives. 2746 area-System.Security CryptographicException on Unix when exporting an empty collection as a PFX This has the same underlying function as a cause as #2743, `PKCS12_create`. Since it never added anything to an authsafe it never created the authsafe stack, and then reports a data validation error because it didn't expect a `NULL` to make it through. When/if we switch to creating PFX files without the helper function's limitations this should go away (but it's a separate problem, and could accidentally not be fixed, so a separate issue). 2751 area-System.Diagnostics Trace.IndentLevel does not behave consistently when used from multiple threads "According to [the documentation](https://msdn.microsoft.com/en-us/library/system.diagnostics.trace.indentlevel), `Trace.IndentLevel` should be stored on a per-thread basis. And [it is](https://github.com/dotnet/corefx/blob/41e203011152581a6c65bb81ac44ec037140c1bb/src/System.Diagnostics.TraceSource/src/System/Diagnostics/TraceInternal.cs#L16), but [a cached version is also kept in each `TraceListener`](https://github.com/dotnet/corefx/blob/41e203011152581a6c65bb81ac44ec037140c1bb/src/System.Diagnostics.TraceSource/src/System/Diagnostics/TraceListener.cs#L18). And since listeners are shared among all threads and the listener-cached version of `IndentLevel` is updated only when `Trace.IndentLevel` changes on some thread, this means that the actual indent level that is printed is the indent level of the last thread that changed it, it doesn't matter what the indent level of the current thread is. Sample application to replicate the issue: ``` c# using System; using System.Diagnostics; using System.Threading; using System.Threading.Tasks; namespace TraceIndentLevel { static class Program { static void Main() { Trace.Listeners.Add(new ConsoleTraceListener()); Task.WaitAll(Task.Run((Action) T1), Task.Run((Action) T2)); } static readonly Barrier barrier = new Barrier(2); static void T1() { Trace.Indent(); Trace.Indent(); Trace.TraceInformation(""Thread 1, IndentLevel should be {0}."", Trace.IndentLevel); barrier.SignalAndWait(); barrier.SignalAndWait(); Trace.TraceInformation(""Thread 1, IndentLevel should be {0}."", Trace.IndentLevel); // this should be a noop, but isn't Trace.IndentLevel = Trace.IndentLevel; Trace.TraceInformation(""Thread 1, IndentLevel should be {0}."", Trace.IndentLevel); } static void T2() { barrier.SignalAndWait(); Trace.TraceInformation(""Thread 2, IndentLevel should be {0}."", Trace.IndentLevel); Trace.Indent(); Trace.TraceInformation(""Thread 2, IndentLevel should be {0}."", Trace.IndentLevel); barrier.SignalAndWait(); } } } ``` On my computer (using desktop .Net, but I believe core will behave the same), this prints: ``` TraceIndentLevel.exe Information: 0 : Thread 1, IndentLevel should be 2. TraceIndentLevel.exe Information: 0 : Thread 2, IndentLevel should be 0. TraceIndentLevel.exe Information: 0 : Thread 2, IndentLevel should be 1. TraceIndentLevel.exe Information: 0 : Thread 1, IndentLevel should be 2. TraceIndentLevel.exe Information: 0 : Thread 1, IndentLevel should be 2. ``` What I would expect is that the value returned by `Trace.IndentLevel` and the actual indent level matched: ``` TraceIndentLevel.exe Information: 0 : Thread 1, IndentLevel should be 2. TraceIndentLevel.exe Information: 0 : Thread 2, IndentLevel should be 0. TraceIndentLevel.exe Information: 0 : Thread 2, IndentLevel should be 1. TraceIndentLevel.exe Information: 0 : Thread 1, IndentLevel should be 2. TraceIndentLevel.exe Information: 0 : Thread 1, IndentLevel should be 2. ``` " 2755 area-System.Reflection Downgrade Reflection.Metadata to Immutable Collections 1.1.36 Can't use the latest version w/o updating VS. 2756 area-Microsoft.CSharp Need to find a way to run tests involving expression trees in interpreter mode "This involves: Linq.Expressions System.Dynamic.Runtime Microsoft.CSharp What complicates matters is that Linq.Expressions must be rebuilt with ""isInterpreting=true"" to switch from compiling to interpreting. " 2757 area-System.IO DirectoryInfo.GetFiles killed by privileged folders The seemingly useful DirectoryInfo.GetFiles method (with the SearchOption.AllDirectories parameter) is unreliable because any access control check failure causes it to bomb out with an exception rather than returning files that _are_ accessible to the user. For instance, call this on any drive root from a non-elevated process and you'll get: `Access to the path 'D:\$RECYCLE.BIN\S-1-5-18' is denied. Type: System.UnauthorizedAccessException Source: mscorlib at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileSystemEnumerableIterator``1.AddSearchableDirsToStack(SearchData localSearchData) at System.IO.FileSystemEnumerableIterator``1.MoveNext() at System.Collections.Generic.List``1..ctor(IEnumerable``1 collection) at System.IO.DirectoryInfo.InternalGetFiles(String searchPattern, SearchOption searchOption)` The API should be smart enough to skip enumerating folders to which it doesn't have permission, or it should offer some sort of callback or parameter to permit doing so. 2758 area-System.Net Enabling cookie container in HTTP xplat ``` modified: ../../Common/src/Interop/Unix/libcurl/Interop.libcurl_types.cs modified: System/Net/Http/Unix/CurlHandler.cs modified: System/Net/Http/Unix/HttpClientHandler.Unix.cs ``` Enabled cookie container & manual cookie support in HTTP. 2760 area-System.Net HttpClient response stream.Read/ReadAsync sporadically returns 0 for a non empty response We see it both in WCF streaming scenarios (receiving a stream from WCF service) and in simple repros like: ``` var c = (await client.PostAsync(url, new StreamContent(new MyStream()))).Content; var stream = await c.ReadAsStreamAsync(); byte[] buff = new byte[4096]; var bytesread = await stream.ReadAsync(buff, 0, 4096); // bytesread is sporadically 0 even though the response does contain data ``` About a half of WCF streaming scenarios fail because of this bug 2761 area-System.Net Unhandled System.Net.Http.WinHttpException in WCF streaming scenarios This blocks WCF streaming on Project K as it happens within first several seconds of single-threaded(!) stress run: ``` 0:020> !pe Exception object: 180d3de4 Exception type: System.Net.Http.WinHttpException Message: Unknown error (0x2ef2) InnerException: StackTrace (generated): SP IP Function 0EACED5C 0890E8FF System_Net_Http!System.Net.Http.WinHttpHandler.SetStatusCallback(SafeWinHttpHandle, WINHTTP_STATUS_CALLBACK)+0x57 0EACED70 0890D4C9 System_Net_Http!System.Net.Http.WinHttpHandler+d__1.MoveNext()+0x779 0EACEFE4 0781C695 mscorlib_ni!System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c__DisplayClass2.b__5(System.Object)+0x35 0EACEFEC 0781F4DB mscorlib_ni!System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(System.Object)+0x43 0EACEFF4 077C4FAD mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)+0x9d 0EACF02C 0781F48B mscorlib_ni!System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()+0x43 0EACF03C 0781FA25 mscorlib_ni!System.Threading.ThreadPoolWorkQueue.Dispatch()+0x1e9 0EACF08C 078DB3CA mscorlib_ni!System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()+0xa ``` The exception results in the following second-chance exception in the process: ``` 0:020> kL # ChildEBP RetAddr 00 0eacee0c 0fb2c37f KERNELBASE!RaiseException+0x48 01 0eaceea8 0fbb6bb3 coreclr!RaiseTheExceptionInternalOnly+0x1d9 02 0eacef28 077c4fad coreclr!IL_Rethrow+0x6a 03 0eacf020 0781f48b mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)+0x9d 04 0eacf034 0781fa25 mscorlib_ni!System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()+0x43 05 0eacf084 078db3ca mscorlib_ni!System.Threading.ThreadPoolWorkQueue.Dispatch()+0x1e9 06 0eacf094 0fda302f mscorlib_ni!System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()+0xa 07 0eacf094 0fb77a92 coreclr!CallDescrWorkerInternal+0x34 08 0eacf0c8 0fb77d01 coreclr!CallDescrWorkerWithHandler+0x60 09 0eacf134 0fc65182 coreclr!MethodDescCallSite::CallTargetWorker+0x158 0a (Inline) -------- coreclr!MethodDescCallSite::Call_RetBool+0xb 0b 0eacf1b4 0fb27f3e coreclr!QueueUserWorkItemManagedCallback+0x23 0c 0eacf1c8 0fb27f91 coreclr!ManagedThreadBase_DispatchInner+0x5c 0d 0eacf25c 0fb2828e coreclr!ManagedThreadBase_DispatchMiddle+0x4e 0e 0eacf2c0 0fb28340 coreclr!ManagedThreadBase_DispatchOuter+0x66 0f 0eacf2e4 0fc18912 coreclr!ManagedThreadBase_FullTransitionWithAD+0x2f 10 (Inline) -------- coreclr!ManagedThreadBase::ThreadPool+0x10 11 0eacf388 0fbfbaf7 coreclr!ManagedPerAppDomainTPCount::DispatchWorkItem+0x99 12 0eacf39c 0fbfc312 coreclr!ThreadpoolMgr::ExecuteWorkRequest+0x31 13 0eacf404 0fb23702 coreclr!ThreadpoolMgr::WorkerThreadStart+0x391 14 0eacf7a8 76f97c04 coreclr!Thread::intermediateThreadProc+0x4d 15 0eacf7bc 773ead1f KERNEL32!BaseThreadInitThunk+0x24 16 0eacf804 773eacea ntdll!__RtlUserThreadStart+0x2f 17 0eacf814 00000000 ntdll!_RtlUserThreadStart+0x1b ``` 2762 area-System.Net Fast native memory leak in WinHttpHandler on corefx The leak is super-fast - 1GB+/minute and completely blocks WCF streaming scenarios. Here are the top offending call stacks: ``` ntdll!RtlAllocateHeap+14D (d:\blue\minkernel\ntos\rtl\heap.c, 1872) msvcrt!malloc+90 (d:\9147\minkernel\crts\crtw32\heap\malloc.c, 285) webio!zcalloc+20 (d:\9147\inetcore\lib\zlib12\zutil.c, 314) webio!updatewindow+39 (d:\9147\inetcore\lib\zlib12\inflate.c, 337) webio!inflate+12D7 (d:\9147\inetcore\lib\zlib12\inflate.c, 1137) webio!Decompress+56 (d:\9147\inetcore\lib\zlib12\iel1_zlib12_wrapper\zlib_wrapper.c, 218) webio!WapDecompProcessCompressedData+33 (d:\9147\net\webio\util\decompression.c, 510) webio!WaDecompDataNeeded+1C (d:\9147\net\webio\util\decompression.c, 580) webio!WapHandleParsedHttpResponse+1D74A (d:\9147\net\webio\http\recvresp.c, 1656) webio!WapHttpConnectionReceiveCompletion+258 (d:\9147\net\webio\http\httpconn.c, 2974) webio!WapTcpReceiveCompletionRoutine+89 (d:\9147\net\webio\conn\tcpconn.c, 3538) webio!WapTcpThreadPoolCompletionRoutine+3F (d:\9147\net\webio\conn\tcpconn.c, 822) webio!WapTpIoCompletionRoutine+B2 (d:\9147\net\webio\util\thrdpool.c, 370) KERNELBASE!BasepTpIoCallback+3A (d:\9147\minkernel\threadpool\kernel32\threadpool.c, 213) ntdll!TppIopExecuteCallback+D6 (d:\blue\minkernel\threadpool\ntdll\io.c, 371) ntdll!TppWorkerThread+52A (d:\blue\minkernel\threadpool\ntdll\worker.c, 1023) KERNEL32!BaseThreadInitThunk+24 (d:\9147\base\win32\client\thread.c, 78) ntdll!__RtlUserThreadStart+2F (d:\blue\minkernel\ntdll\rtlstrt.c, 1029) ntdll!_RtlUserThreadStart+1B (d:\blue\minkernel\ntdll\rtlstrt.c, 944) --- ntdll!RtlAllocateHeap+14D (d:\blue\minkernel\ntos\rtl\heap.c, 1872) webio!WaDecompCreateContext+AB (d:\9147\net\webio\util\decompression.c, 219) webio!WapHandleParsedHttpResponse+1D61A (d:\9147\net\webio\http\recvresp.c, 1496) webio!WapHttpConnectionReceiveCompletion+258 (d:\9147\net\webio\http\httpconn.c, 2974) webio!WapTcpReceiveCompletionRoutine+89 (d:\9147\net\webio\conn\tcpconn.c, 3538) webio!WapTcpThreadPoolCompletionRoutine+3F (d:\9147\net\webio\conn\tcpconn.c, 822) webio!WapTpIoCompletionRoutine+B2 (d:\9147\net\webio\util\thrdpool.c, 370) KERNELBASE!BasepTpIoCallback+3A (d:\9147\minkernel\threadpool\kernel32\threadpool.c, 213) ntdll!TppIopExecuteCallback+D6 (d:\blue\minkernel\threadpool\ntdll\io.c, 371) ntdll!TppWorkerThread+52A (d:\blue\minkernel\threadpool\ntdll\worker.c, 1023) KERNEL32!BaseThreadInitThunk+24 (d:\9147\base\win32\client\thread.c, 78) ntdll!__RtlUserThreadStart+2F (d:\blue\minkernel\ntdll\rtlstrt.c, 1029) ntdll!_RtlUserThreadStart+1B (d:\blue\minkernel\ntdll\rtlstrt.c, 944) ----- ntdll!RtlAllocateHeap+14D (d:\blue\minkernel\ntos\rtl\heap.c, 1872) msvcrt!malloc+90 (d:\9147\minkernel\crts\crtw32\heap\malloc.c, 285) webio!zcalloc+20 (d:\9147\inetcore\lib\zlib12\zutil.c, 314) webio!inflateInit2_+4F (d:\9147\inetcore\lib\zlib12\inflate.c, 164) webio!CreateDecompression+61 (d:\9147\inetcore\lib\zlib12\iel1_zlib12_wrapper\zlib_wrapper.c, 175) webio!WapHandleParsedHttpResponse+1D61A (d:\9147\net\webio\http\recvresp.c, 1496) webio!WapHttpConnectionReceiveCompletion+258 (d:\9147\net\webio\http\httpconn.c, 2974) webio!WapTcpReceiveCompletionRoutine+89 (d:\9147\net\webio\conn\tcpconn.c, 3538) webio!WapTcpThreadPoolCompletionRoutine+3F (d:\9147\net\webio\conn\tcpconn.c, 822) webio!WapTpIoCompletionRoutine+B2 (d:\9147\net\webio\util\thrdpool.c, 370) KERNELBASE!BasepTpIoCallback+3A (d:\9147\minkernel\threadpool\kernel32\threadpool.c, 213) ---- ntdll!RtlAllocateHeap+14D (d:\blue\minkernel\ntos\rtl\heap.c, 1872) winhttp!WinHttpOpenRequestInternal+7C1 (d:\9147\net\winhttp\http\open.cxx, 717) winhttp!WinHttpOpenRequest+1F9 (d:\9147\net\winhttp\http\open.cxx, 881) ---- ntdll!RtlAllocateHeap+14D (d:\blue\minkernel\ntos\rtl\heap.c, 1872) webio!WapCreateHttpRequest+C1 (d:\9147\net\webio\http\httpreq.c, 1139) webio!WebCreateHttpRequest+183 (d:\9147\net\webio\http\httpreq.c, 3192) winhttp!HTTP_USER_REQUEST::_SysSendRequest+D32 (d:\9147\net\winhttp\httpcore\usrreq.cpp, 2347) winhttp!HTTP_USER_REQUEST::_SendRequestWithDrainComplete+1A0 (d:\9147\net\winhttp\httpcore\usrreq.cpp, 1199) winhttp!HTTP_USER_REQUEST::SendRequest+3F3 (d:\9147\net\winhttp\httpcore\usrreq.cpp, 1529) winhttp!WinHttpSendRequest+4B9 (d:\9147\net\winhttp\api\sendapi.cxx, 265) ``` 2763 area-System.Net The result of the arithmetic + operation in Write method could end up… … being less than zero if the sum of the operands combined is greater than int.MaxValue. As a consequence of this integer overflow the intended check might be bypassed leading to unexpected consequences. This change adds and aditional cast to long to avoid this overflow. 2764 area-System.IO NamedPipeServerStream constructor allows MaxNumberOfServerInstances > 1 The constructor for NamedPipeServerStream allows MaxNumberOfServerInstances > 1 and we even do the same validity checks (between 1-255) as we do on Windows. However, Unix Named pipes don't support more than one connection and an exception is thrown when more are attempted. 2765 area-System.Collections ExpandPrime will throw when near MaxPrimeArrayLength Call ExpandPrime in HashHelpers.cs and HashTable.cs with MaxPrimeArrayLength + 1. The function looks like this: public static int ExpandPrime(int oldSize) { int newSize = 2 \* oldSize; ``` if ((uint)newSize > MaxPrimeArrayLength && MaxPrimeArrayLength > oldSize) { return MaxPrimeArrayLength; } return GetPrime(newSize); ``` } MaxPrimeArrayLength + 1 become -2097156 which, when cast to uint becomes 4292870140 which passes the 'if' condition and GetPrime is then called with a negative value. This caused GetPrime to: throw new ArgumentException(SR.Arg_HTCapacityOverflow); proposed solution: public static int ExpandPrime(int oldSize) { int newSize = 2 \* oldSize; ``` // newSize will be negative if it overflows so we return the largest possible instead if (newSize < 0 || oldSize > MaxPrimeArrayLength) { return MaxPrimeArrayLength; } return GetPrime(newSize); ``` } 2768 area-System.Reflection CoreCLR: TryGetRawMetadata "# Scenario Roslyn Scripting API provides applications with the capability to evaluate C# and VB code snippets. For example, ``` C# using Microsoft.CodeAnalysis.Scripting.CSharp; public class Globals { public int x; public int y; } class Program { public static void Main() { var globals = new Globals() { x = 1, y = 2 }; var options = ScriptOptions.Default.WithReferences(typeof(MyMath).Assembly); var script = CSharpScript.Create(""MyMath.CalculateStuff(x, y)"", options); Console.WriteLine(script.RunAsync(globals).Result); } } ``` In the example above `MyMath` is a type defined in an assembly referenced by and deployed with the program. The script is compiled with the following metadata references: mscorlib, the assembly containing the `Globals` type and an assembly containing `MyMath` class. All these assemblies are implementation (runtime) assemblies, not contracts. Since we are compiling scripts at runtime contracts are not relevant. The application might also run on a machine that doesn’t have any SDK. It is natural for scripting to be able to execute virtually any (public) runtime method available in any loaded assembly. Just like Reflection invocation. On Desktop CLR we use Assembly.Location to find the metadata image for given assembly. On Core CLR this property is not available and there is no other way to get the metadata of a given runtime assembly. # Proposed API We propose that the metadata of a runtime assembly are exposed thru new APIs in a form that is efficiently readable using MetadataReader of System.Reflection.Metadata library. Since metadata might not be available for all assemblies (such as AssemblyBuilder, .NET Native images, etc.) the API is allowed to return false for such assemblies. Roslyn scripting API would then report an error that the script can’t reference specified assembly. ``` C# namespace System.Reflection.Metadata { public static class AssemblyExtensions { public unsafe static bool TryGetRawMetadata(this Assembly assembly, out byte* blob, out int length); } } ``` **Lifetime considerations** The pointer would stay valid as long as the `AssemblyLoadContext` with which is the assembly associated is alive. It is the responsibility of the caller to keep the Assembly object alive while accessing the metadata blob. # Considered alternatives It was suggested that Roslyn uses Reflection API to read the metadata. There are multiple issues with such approach. 1) Expected bad performance (both working set and time) Reflection is built on top of IMetadataImport. It was measured that using IMetadataImport to reader metadata from managed code is an order of magnitude slower than using S.R.M. MetadataReader, which was not an acceptable perf hit for the compilers. Reflection API implementation adds additional memory and time overhead. Although the number of referenced assemblies is relatively low in most scripting scenarios reducing the compile time of scripts to minimum is crucial for good user experience. 2) Using Reflection APIs to read namespaces and types To get a list of namespaces in given assembly one needs to load all types in the assembly. Loading a type may trigger additional assembly loads as well (those of the base types, interfaces). If a type can’t be loaded (perhaps a dependency is missing) we get a loader exception. We can extract the loaded types from the exception, but what if the app makes the dependency available just before it is about to actually start using the type? Our model is immutable, we can’t add a type/namespace into an existing assembly symbol. 3) Missing APIs Brief analysis identified missing APIs that the compiler depends on and that are not present in DNX contracts: for example, type forwarders, GetOptionalCustomModifiers, GetRequiredCustomModifiers, class layout. Besides missing APIs Reflection performs post-processing on the data it receives from IMetadataImport that might render the values useless to the compiler. 4) Roslyn-Reflection interop We have prototyped an abstraction of Reflection symbol model to incorporate interop with Reflection runtime types. The abstraction introduces quite a lot of complexity and indirections in already complex code, which significantly increases maintenance cost of that particular area of the compiler. Testing cost would also need to be considered. We would need to effectively run most of our compiler tests twice. # Updates 1) Decided to move this to `System.Reflection.Metadata` namespace. " 2769 area-System.Globalization System.Globalization: Collation Implement the [CompareInfo PAL](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/corefx/System/Globalization/CompareInfo.Unix.cs) on top of ICU. This should give us real collation support instead of our ASCII only collation. 2770 area-System.Globalization System.Globalization: Locale Data Implement the PAL's from [CultureData](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/corefx/System/Globalization/CultureData.Unix.cs) and [CultureInfo](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/corefx/System/Globalization/CultureInfo.Unix.cs). 2772 area-System.Globalization System.Globalization: Casing Implement the [TextInfo](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/corefx/System/Globalization/TextInfo.Unix.cs) PAL. 2773 area-System.Globalization System.Globalization: Calendars Fill out the PAL implementation of [CalendarData.cs](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/corefx/System/Globalization/CalendarData.Unix.cs), [HijriCalendar.cs](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/corefx/System/Globalization/HijriCalendar.Unix.cs) and [JapaneseCalendar.cs](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/corefx/System/Globalization/JapaneseCalendar.Unix.cs) 2774 area-System.Globalization System.Globalization: Encodings We have stubbed out [EncodingTable.cs](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/src/System/Globalization/EncodingTable.Unix.cs) and [EncodingDataItem.cs](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/src/System/Globalization/EncodingDataItem.Unix.cs) for cross platform, but it is not clear the implementation is correct or if we need to do more. I think the big issue here will be understanding if System.Text.Encoding.Codepages and the EncodingProvider model here reduces the scope of work that we need to. 2777 area-System.Globalization System.Globalization.Extensions: Normalization We need to implement [StringNormalizationExtensions.cs](https://github.com/dotnet/corefx/blob/master/src/Common/src/System/StringNormalizationExtensions.Unix.cs) for Linux. The major difference between ICU and Win32 is that Win32 throws an exception when the input is not a valid UTF-16 encoded string (e.g. mismatched surrogates) whereas ICU does not. 2778 area-System.Security Add ECDsa to System.Security.Cryptography suite - `ECDsa` in System.Security.Cryptography.Algorithms - `ECDsaCng : ECDsa` in System.Security.Cryptography.Cng - `ECDsaOpenSsl : ECDsa` in System.Security.Cryptography.OpenSsl - Some tests (Common\test\Cryptography\AlgorithmImplementations\ECDsa) - Other artifacts required to pull this off. - Like the contract assembly changes 2779 area-System.Globalization System.Globalization.Extensions: IDNA We need to support [IdnMapping.cs](https://github.com/dotnet/corefx/blob/master/src/Common/src/System/Globalization/IdnMapping.Unix.cs) for IDNA2008 on Unix. This is straightforward on-top of ICU, but many of our existing ICU tests assume either ICU2003 or ICU2008 as implemented by Win32. There are some differences between 2008 and 2003 in ICU for compatibility, especially around how windows works when UseStd3Rules is false. In speaking with NLS, they think it may make sense for us to just follow the 2008 standard. 2782 area-System.Diagnostics Add a way to create a suspended Process It seems that there is no way to create a suspended `Process` and resume it later. Ability to create a suspended process is useful when you want to run the process and its children in a job object. If the process isn't created as suspended it may create child processes before it is assigned to the job object which means that these child processes won't be assigned to the job object. 2784 area-System.Net Enabled max redirection in HTTP xplat ``` modified: ../../Common/src/Interop/Unix/libcurl/Interop.libcurl_types.cs modified: System/Net/Http/Unix/CurlHandler.cs modified: System/Net/Http/Unix/HttpClientHandler.Unix.cs ``` Enabled max redirection support in HTTP xplat 2785 area-System.Net Server Credential Support for CurlHandler This check-in integrates curlHandler with the credential support of libcurl. 2787 area-System.Net Use common class to set property default in winHttp & xplatHttp > stephentoub :: There are a bunch of these default values that are being copied from the WinHttp handler implementation. It's great that we're using the same defaults, but it's error prone that we're doing so via copy-and-paste. I see a whole bunch of constants/defaults in WinHttp at https://github.com/dotnet/corefx/blob/master/src/System.Net.Http.WinHttpHandler/src/System/Net/Http/WinHttpHandler.cs#L93-L146 .could we somehow factor out these constants/default values into a shared source file that's included in both the Windows and Unix implementation, such that this then becomes: > > ``` C# > private int _maxAutomaticRedirections = HttpHandlerDefaults.MaxAutomaticRedirections; > ``` > > instead of: > > ``` C# > private int _maxAutomaticRedirections = 50; > ``` > > davidsh :: Yes, one could create a shared source file in src\Common\src\System\Net\Http... that defined an internal static class (HttpHandlerDefaults) with those constants defined as public fields. Then that file would be used in the multiple .CSPROJ files to bring in the values. 2791 area-System.Net Unsupported address family assert in System.Net.SocketAddressPal.SetPort on Linux in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst_prtest/1767/console ``` 11:17:17 Discovering: System.Net.Primitives.Linux.Unit.Tests 11:17:17 Discovering: System.Net.Primitives.Functional.Tests 11:17:17 Discovering: System.Net.Primitives.Linux.Pal.Tests 11:17:18 Discovered: System.Net.Primitives.Functional.Tests 11:17:18 Discovered: System.Net.Primitives.Linux.Unit.Tests 11:17:18 Discovered: System.Net.Primitives.Linux.Pal.Tests 11:17:18 Starting: System.Net.Primitives.Functional.Tests 11:17:18 Starting: System.Net.Primitives.Linux.Unit.Tests 11:17:18 Starting: System.Net.Primitives.Linux.Pal.Tests 11:17:18 ---- DEBUG ASSERTION FAILED ---- 11:17:18 ---- Assert Short Message ---- 11:17:18 Unsupported address family 11:17:18 ---- Assert Long Message ---- 11:17:18 11:17:18 at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) 11:17:18 at System.Environment.get_StackTrace() 11:17:18 at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) 11:17:18 at System.Diagnostics.Debug.Fail(String message) 11:17:18 at System.Net.SocketAddressPal.SetPort(Byte[] buffer, UInt16 port) 11:17:18 at System.Net.Primitives.PalTests.SocketAddressPalTests.<>c__DisplayClass6.b__4() 11:17:18 at Xunit.Assert.RecordException(Action testCode) 11:17:18 at Xunit.Assert.ThrowsAny[T](Action testCode) 11:17:18 at System.Net.Primitives.PalTests.SocketAddressPalTests.Port_Get_Set_Throws() 11:17:18 at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) 11:17:18 at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) 11:17:18 at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) 11:17:18 at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass45_1.<b__1>d.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass45_1.b__1() 11:17:18 at Xunit.Sdk.ExecutionTimer.d__4.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) 11:17:18 at Xunit.Sdk.ExceptionAggregator.d__9.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) 11:17:18 at Xunit.Sdk.TestInvoker`1.d__45.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestInvoker`1.InvokeTestMethodAsync(Object testClassInstance) 11:17:18 at Xunit.Sdk.TestInvoker`1.<b__44_0>d.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestInvoker`1.b__44_0() 11:17:18 at Xunit.Sdk.ExceptionAggregator.d__10`1.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExceptionAggregator.RunAsync[T](Func`1 code) 11:17:18 at Xunit.Sdk.XunitTestRunner.d__4.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.XunitTestRunner.InvokeTestAsync(ExceptionAggregator aggregator) 11:17:18 at Xunit.Sdk.ExceptionAggregator.d__10`1.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExceptionAggregator.RunAsync[T](Func`1 code) 11:17:18 at Xunit.Sdk.TestRunner`1.d__43.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestCaseRunner`1.d__19.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestCaseRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestMethodRunner`1.d__32.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestMethodRunner`1.RunTestCasesAsync() 11:17:18 at Xunit.Sdk.TestMethodRunner`1.d__31.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestMethodRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestClassRunner`1.d__38.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestClassRunner`1.RunTestMethodsAsync() 11:17:18 at Xunit.Sdk.TestClassRunner`1.d__37.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestClassRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.d__28.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.RunTestClassesAsync() 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.d__27.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.RunAsync() 11:17:18 at System.Threading.Tasks.Task`1.InnerInvoke() 11:17:18 at System.Threading.Tasks.Task.Execute() 11:17:18 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) 11:17:18 at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot) 11:17:18 at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution) 11:17:18 at Xunit.Sdk.MaxConcurrencySyncContext.<>c__DisplayClass9_0.b__0(Object _) 11:17:18 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) 11:17:18 at Xunit.Sdk.MaxConcurrencySyncContext.WorkerThreadProc() 11:17:18 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) 11:17:18 ---- DEBUG ASSERTION FAILED ---- 11:17:18 ---- Assert Short Message ---- 11:17:18 Unsupported address family 11:17:18 ---- Assert Long Message ---- 11:17:18 11:17:18 at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) 11:17:18 at System.Environment.get_StackTrace() 11:17:18 at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) 11:17:18 at System.Diagnostics.Debug.Fail(String message) 11:17:18 at System.Net.SocketAddressPal.GetPort(Byte[] buffer) 11:17:18 at System.Net.Primitives.PalTests.SocketAddressPalTests.<>c__DisplayClass6.b__5() 11:17:18 at Xunit.Assert.RecordException(Func`1 testCode) 11:17:18 at Xunit.Assert.ThrowsAny[T](Func`1 testCode) 11:17:18 at System.Net.Primitives.PalTests.SocketAddressPalTests.Port_Get_Set_Throws() 11:17:18 at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) 11:17:18 at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) 11:17:18 at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) 11:17:18 at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass45_1.<b__1>d.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass45_1.b__1() 11:17:18 at Xunit.Sdk.ExecutionTimer.d__4.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) 11:17:18 at Xunit.Sdk.ExceptionAggregator.d__9.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) 11:17:18 at Xunit.Sdk.TestInvoker`1.d__45.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestInvoker`1.InvokeTestMethodAsync(Object testClassInstance) 11:17:18 at Xunit.Sdk.TestInvoker`1.<b__44_0>d.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestInvoker`1.b__44_0() 11:17:18 at Xunit.Sdk.ExceptionAggregator.d__10`1.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExceptionAggregator.RunAsync[T](Func`1 code) 11:17:18 at Xunit.Sdk.XunitTestRunner.d__4.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.XunitTestRunner.InvokeTestAsync(ExceptionAggregator aggregator) 11:17:18 at Xunit.Sdk.ExceptionAggregator.d__10`1.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.ExceptionAggregator.RunAsync[T](Func`1 code) 11:17:18 at Xunit.Sdk.TestRunner`1.d__43.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestCaseRunner`1.d__19.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestCaseRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestMethodRunner`1.d__32.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestMethodRunner`1.RunTestCasesAsync() 11:17:18 at Xunit.Sdk.TestMethodRunner`1.d__31.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestMethodRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestClassRunner`1.d__38.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestClassRunner`1.RunTestMethodsAsync() 11:17:18 at Xunit.Sdk.TestClassRunner`1.d__37.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestClassRunner`1.RunAsync() 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.d__28.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.RunTestClassesAsync() 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.d__27.MoveNext() 11:17:18 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine) 11:17:18 at Xunit.Sdk.TestCollectionRunner`1.RunAsync() 11:17:18 at System.Threading.Tasks.Task`1.InnerInvoke() 11:17:18 at System.Threading.Tasks.Task.Execute() 11:17:18 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) 11:17:18 at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot) 11:17:18 at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution) 11:17:18 at Xunit.Sdk.MaxConcurrencySyncContext.<>c__DisplayClass9_0.b__0(Object _) 11:17:18 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) 11:17:18 at Xunit.Sdk.MaxConcurrencySyncContext.WorkerThreadProc() 11:17:18 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) ``` 2793 area-System.Net Massive and fast handle leak in WCF stress scenarios 0:102> !handle 0 0 Type Count Event 26762 Key 20745 Here are the characteristic call stacks: ``` Handle = 0x000000000002ea84 - OPEN Thread ID = 0x0000000000002d74, Process ID = 0x0000000000004528 0x00007ff8c74c225a: ntdll!ZwOpenKeyEx+0x000000000000000a 0x00007ff8c47c3799: KERNELBASE!LocalBaseRegOpenKey+0x00000000000001fa 0x00007ff8c47cf532: KERNELBASE!RegOpenKeyExInternalA+0x0000000000000132 0x00007ff8c47cf5e9: KERNELBASE!RegOpenKeyExA+0x0000000000000019 0x00007ff8c0b82b1c: winhttp!OpenIeKey+0x000000000000005c 0x00007ff8c0b82c08: winhttp!OpenProxySettingsKeysForRead+0x0000000000000080 0x00007ff8c0b998bb: winhttp!INTERNET_SESSION_HANDLE_OBJECT::LoadAutomaticProxyResolvers+0x000000000000019b 0x00007ff8c0b730e5: winhttp!INTERNET_SESSION_HANDLE_OBJECT::SetProxySettings+0x0000000000044ed5 0x00007ff8c0b2d95a: winhttp!WinHttpSetOptionInternal+0x0000000000000397 0x00007ff8c0b2e9e6: winhttp!WinHttpOpen+0x000000000000027a 0x00007ff841eeccbd: System_Net_Http!DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr, UInt32, System.String, System.String, Int32)+0x000000000000010d 0x00007ff841eec8b3: System_Net_Http!System.Net.Http.WinHttpHandler.EnsureSessionHandleExists(RequestState)+0x00000000000000e3 0x00007ff841eebba1: System_Net_Http!System.Net.Http.WinHttpHandler+d__1.MoveNext()+0x00000000000001f1 -------------------------------------- Handle = 0x000000000002dd10 - OPEN Thread ID = 0x00000000000027b4, Process ID = 0x0000000000004528 0x00007ff8c74c165a: ntdll!ZwCreateEvent+0x000000000000000a 0x00007ff8c47c2ce8: KERNELBASE!CreateEventW+0x0000000000000084 0x00007ff8c0b2ed38: winhttp!INTERNET_SESSION_HANDLE_OBJECT::INTERNET_SESSION_HANDLE_OBJECT+0x00000000000002be 0x00007ff8c0b2e963: winhttp!WinHttpOpen+0x00000000000001f3 0x00007ff8c0bb9afb: winhttp!CreateWinHttpSession+0x0000000000000053 0x00007ff8c0bb2fd9: winhttp!WinHttpClientSession::AcquireSession+0x0000000000000049 0x00007ff8c0bb3e34: winhttp!WinHttpClientResolver::GetProxyForUrlImpl+0x00000000000001a8 0x00007ff8c0bb3a48: winhttp!WinHttpClientResolver::GetProxyForUrlEx+0x0000000000000054 0x00007ff8c0bb50d9: winhttp!WinHttpClientCompletion::StartProxyResolve+0x0000000000000049 0x00007ff8c0b6102d: winhttp!WxProxyManager::OnProcessGetProxyForUrl+0x000000000002668d 0x00007ff8c0bb665b: winhttp!WxProxyManager::GetProxyForUrlImpl+0x00000000000000ef 0x00007ff8c0bb6518: winhttp!WxProxyManager::GetProxyForUrlEx+0x0000000000000054 0x00007ff8c0bb7683: winhttp!WxGetProxyContext::StartProxyResolve+0x0000000000000043 0x00007ff8c0b711ed: winhttp!HTTP_USER_REQUEST::_CallGetProxyForUrl+0x000000000003478d -------------------------------------- -------------------------------------- Handle = 0x000000000000cfa0 - OPEN Thread ID = 0x0000000000002d74, Process ID = 0x0000000000004528 0x00007ff8c74c165a: ntdll!ZwCreateEvent+0x000000000000000a 0x00007ff8c47c2ce8: KERNELBASE!CreateEventW+0x0000000000000084 0x00007ff8c0b2ed38: winhttp!INTERNET_SESSION_HANDLE_OBJECT::INTERNET_SESSION_HANDLE_OBJECT+0x00000000000002be 0x00007ff8c0b2e963: winhttp!WinHttpOpen+0x00000000000001f3 0x00007ff8c0bb9afb: winhttp!CreateWinHttpSession+0x0000000000000053 0x00007ff8c0bb2fd9: winhttp!WinHttpClientSession::AcquireSession+0x0000000000000049 0x00007ff8c0bb3e34: winhttp!WinHttpClientResolver::GetProxyForUrlImpl+0x00000000000001a8 0x00007ff8c0bb3a48: winhttp!WinHttpClientResolver::GetProxyForUrlEx+0x0000000000000054 0x00007ff8c0bb50d9: winhttp!WinHttpClientCompletion::StartProxyResolve+0x0000000000000049 0x00007ff8c0b6102d: winhttp!WxProxyManager::OnProcessGetProxyForUrl+0x000000000002668d 0x00007ff8c0bb665b: winhttp!WxProxyManager::GetProxyForUrlImpl+0x00000000000000ef 0x00007ff8c0bb6518: winhttp!WxProxyManager::GetProxyForUrlEx+0x0000000000000054 0x00007ff8c0bb7683: winhttp!WxGetProxyContext::StartProxyResolve+0x0000000000000043 0x00007ff8c0b711ed: winhttp!HTTP_USER_REQUEST::_CallGetProxyForUrl+0x000000000003478d -------------------------------------- ``` 2794 area-System.IO Fixed test that was leaving temp directories in root folder One of the Extended Path FileSystem tests was leaving temporary test directories in the root folder of the running directory. This small change modified the tests to instead treat the user's temp directory as the root. 2799 area-System.Net SendAsync_SlowGetRequestWithTimedCancellation test failed in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug/999/console 20:01:52 Discovering: System.Net.Http.Unit.Tests 20:01:53 System.Net.Http.WinHttpHandlerUnitTests.WinHttpHandlerTests.SendAsync_SlowGetRequestWithTimedCancellation_ExpectTaskCanceledException [FAIL] 20:01:53 Assert.Throws() Failure 20:01:53 Expected: typeof(System.Threading.Tasks.TaskCanceledException) 20:01:53 Actual: typeof(System.NullReferenceException): Object reference not set to an instance of an object. 20:01:53 Stack Trace: 20:01:53 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Http.WinHttpHandler\tests\UnitTests\FakeInterop.cs(238,0): at Interop.WinHttp.WinHttpQueryHeaders(SafeWinHttpHandle requestHandle, UInt32 infoLevel, String name, StringBuilder buffer, UInt32& bufferLength, IntPtr index) 20:01:53 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Http.WinHttpHandler\src\System\Net\Http\WinHttpHandler.cs(1970,0): at System.Net.Http.WinHttpHandler.GetResponseHeaderStringInfo(SafeWinHttpHandle requestHandle, UInt32 infoLevel) 20:01:53 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Http.WinHttpHandler\src\System\Net\Http\WinHttpHandler.cs(2014,0): at System.Net.Http.WinHttpHandler.ParseResponseHeaders(SafeWinHttpHandle requestHandle, HttpResponseMessage response, Boolean stripEncodingHeaders) 20:01:53 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Http.WinHttpHandler\src\System\Net\Http\WinHttpHandler.cs(1911,0): at System.Net.Http.WinHttpHandler.CreateResponseMessage(SafeWinHttpHandle connectHandle, SafeWinHttpHandle requestHandle, HttpRequestMessage request) 20:01:53 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Http.WinHttpHandler\src\System\Net\Http\WinHttpHandler.cs(1239,0): at System.Net.Http.WinHttpHandler.d__5.MoveNext() 20:01:53 --- End of stack trace from previous location where exception was thrown --- 20:01:53 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 20:01:53 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 20:01:53 at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() 20:01:53 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Http.WinHttpHandler\tests\UnitTests\WinHttpHandlerTest.cs(833,0): at System.Net.Http.WinHttpHandlerUnitTests.WinHttpHandlerTests.<>c__DisplayClassd7.<b__d6>d__d9.MoveNext() 20:01:53 --- End of stack trace from previous location where exception was thrown --- 20:01:53 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 20:01:53 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 20:01:53 Finished: System.Net.Http.WinHttpHandler.Unit.Tests 20:01:53 20:01:53 === TEST EXECUTION SUMMARY === 20:01:53 System.Net.Http.WinHttpHandler.Unit.Tests Total: 115, Errors: 0, Failed: 1, Skipped: 0, Time: 2.519s 2800 area-System.Reflection Assembly.ReflectionOnlyLoad replacement - TypeInfo and family implementation over System.Reflection.Metadata Sometimes you want to use the same reflection model without loading actual types (attributes of course are interesting), but this would be a better version of reflection only load. Roslyn does something similar and can go from PEReader -> ISymbol. /cc @pranavkm @nguerrera @tmat 2801 area-Infrastructure Nice to have: Bin and Packages folders should be placed outside of the git repo The huge `Bin` and `Packages` folders are placed in the root of the git repository at the moment. This makes tools like VSCode impossible to use due to the large quantity of files they need to index. A workaround to use VSCode is to load only the \src folder but that is breaking the Git extension which requires the root of the repo to be loaded. Another reason is using a different partition for binaries that can be quickly formatted when the binaries and packages are not wanted anymore or placing it on a faster disk. 2804 area-System.Globalization System.Globalization: Abbreviated genitive month names won't work on custom DateTimeFormat strings "As found out by @jskeet , abbreviated genitive month names don't seem to work when using the ""MMM"" token on custom datetime format strings The test, written by @jskeet (I'm just copying here from https://github.com/nodatime/nodatime/issues/377): ``` using System; using System.Globalization; using System.Linq; class Test { static void Main() { var culture = (CultureInfo) CultureInfo.InvariantCulture.Clone(); culture.DateTimeFormat.AbbreviatedMonthGenitiveNames = culture.DateTimeFormat.AbbreviatedMonthNames .Select(x => ""Gen"" + x) .ToArray(); culture.DateTimeFormat.MonthGenitiveNames = culture.DateTimeFormat.MonthNames .Select(x => ""Gen"" + x) .ToArray(); Console.WriteLine(new DateTime(1976, 6, 19).ToString(""d MMMM yy"", culture)); Console.WriteLine(new DateTime(1976, 6, 19).ToString(""MMMM yy"", culture)); Console.WriteLine(new DateTime(1976, 6, 19).ToString(""d MMM yy"", culture)); Console.WriteLine(new DateTime(1976, 6, 19).ToString(""MMM yy"", culture)); } } ``` Should output: ``` 19 GenJune 76 June 76 19 GenJun 76 Jun 76 ``` But outputs: ``` 19 GenJune 76 June 76 19 Jun 76 Jun 76 ``` Disregarding the abbreviated genitives (and resorting to regular ones when abbreviated). I haven't found the DateTimeFormat files on GitHub yet (just the tests, which don't cover this), so I can't issue a request or test it, but on the reference source: http://referencesource.microsoft.com/#mscorlib/system/globalization/datetimeformat.cs,578 We got this: ``` if ((dtfi.FormatFlags & DateTimeFormatFlags.UseGenitiveMonth) != 0 && tokenLen >= 4) { result.Append( dtfi.internalGetMonthName( month, IsUseGenitiveForm(format, i, tokenLen, 'd')? MonthNameStyles.Genitive : MonthNameStyles.Regular, false)); ``` Which doesn't take MMM into account (because of the `tokenLen >= 4` conditional). If I'm not mistaken, it should look like this: ``` if ((dtfi.FormatFlags & DateTimeFormatFlags.UseGenitiveMonth) != 0 && tokenLen >= 3 /* Work for MMM and MMMM */ ) { result.Append( dtfi.internalGetMonthName( month, IsUseGenitiveForm(format, i, tokenLen, 'd')? MonthNameStyles.Genitive : MonthNameStyles.Regular, tokenLen == 3 /* Abbreviated if MMM */)); ``` " 2805 area-System.Net Introducing checks for libcurl features Runtime and load-time checks to ensure that the features user is relying on are available in the libcurl library loaded 2810 area-System.Net Moving shared default between winHttp & Unix Handler to a seperate common file ``` new file: ../../Common/src/System/Net/Http/HttpHandlerDefaults.cs modified: ../../System.Net.Http.WinHttpHandler/src/System/Net/Http/WinHttpHandler.cs modified: System.Net.Http.csproj modified: System/Net/Http/Unix/CurlHandler.cs ``` Moved shared default to common file. Addressing issue #2787 . 2813 area-Infrastructure Investigate a better way to handle * dependencies "We're inconsistent across the repo and buildtools. Some places we use -\* others we use -beta-_. It seems that both have the draw back of not permitting an upgrade to latest stable version, or moving past any changes in the major.minor.build version. In most places the semantics we want is either 4.0._: latest of a given API version or 4.*: latest API of a major release. In conjunction with this we need to experiment with how to use policy to impact the behavior of restore. In some cases we want ""minimum that satisfies the constraint"" in others we want ""latest that satisfies the constraint"". /cc @weshaggard " 2817 area-Infrastructure Investigate: Partial facades should build against stable packages Today partial facades are building against live dependencies via P2P reference for other contracts. This means that they always need to reference the latest version of their contract dependencies. We should see if we can break this to allow the partial facades to version more independently. 2833 area-System.Net Credential Support in Unix HTTP Handler This checkin integrates CurlHandler to the credential support available in libcurl 2836 area-System.IO Add method for calculating relative paths It would be great if this super common functionality could be added to the path class. Using URI feels hacky to do this anyways. Related https://github.com/dotnet/corefx/issues/1745 2837 area-System.Threading SemaphoreSlim Dispose thread safety The documentation of SemaphoreSlim says 'Dispose should only be used when all other operations have completed'. In practice, for me, I don't bother to Dispose the SemaphoreSlim because I would need to add synchronization to Dispose of this synchronization class. See also, this [question on stackoverflow](http://stackoverflow.com/questions/32035891/dispose-a-semaphoreslim-while-waitasync). Can SemaphoreSlim be extended so pending operations on other threads throw ObjectDisposedException? Related to this, I noticed some of the other synchronization classes in System.Threading do not properly document the thread safety behavior of their Dispose method. Furthermore, the reference source of ReaderWriterLockSlim shows a check in the Dispose method which throws a SynchronizationLockException when the Dispose conditions are not met. SemaphoreSlim and other classes could do the same. 2840 area-System.Net Enabling POST method options 2844 area-System.Linq Implement IList in Linq responses when possible There is code throughout .NET that accepts arguments of `IEnumerable` and has an optimised path for `IList` and/or `ICollection`. Some methods of `System.Linq.Enumerable` would be examples, but also elsewhere such as some collections' constructors. The effect upon `.ToList()`, `.ToArray()` and any operation in `Enumerable` that requires buffering are perhaps particularly of note here. Some methods of `Enumerable` have results which are conceptually lists. E.g. `Repeat` returns a sequence of a specified size where all items are the same, which can be modelled as a read-only list. Some methods of `Enumerable` have results which can be conceptually lists in certain conditions. E.g. If `Select` is called on a source that is itself a list, then the result would be a read-only list of the same size as the source list, and a different type. Returning such list types when possible would increase the times that the existing fast-paths for list types are made use of. At the same time, some such changes would add further such cases, potentially allowing chains of operations that cannot currently hit such fast-paths to do so. 2849 area-System.Security Add support for importing PKCS7 files on Unix 2851 area-Meta Port legacy LINQ tests to Git/XUnit "Current active test coverage for LINQ is not very great. While sources were ported and published, many old legacy tests has been left behind. It is not an easy task to port them to Git/XUnit, but mostly just because of the test volume and because they are not XUnit tests. It would be nice to port them over. Also it would be preferable to port these tests over writing new ones, because the old tests are by definition ""correct"". I.E. any deviation from the behavior guarded by those tests could be a compatibility issue. I have collected and pushed a lot of old legacy tests for LINQ to https://github.com/VSadov/corefx/tree/UnportedLegacyTests branch. Actual tests are at: https://github.com/VSadov/corefx/tree/UnportedLegacyTests/src/System.Linq/LegacyTestsToPort/LINQ " 2857 area-System.Net HttpClient Timeout Confusion The HttpClient from the System.Net.Http package has Timeout property that defaults to a 100 seconds which as I read through the code just means how long till the task is cancelled. The underlying windows handler WinHttpHandler has the timeout properties ConnectionTimeout, SendTimeout, ReceiveHeadersTimeout and ReceiveDataTimeout which default to 30 seconds except for ConnectionTimeout which defaults to 60 seconds. If you have an operations that takes say 40 seconds before it return back any data it will always timeout after 30 seconds. Only way to get around this is to code against the platform specific WinHttpHandler and set the timeout properties. Am I missing something about how to set timeout in a platform agnositic approach? Here is the code I wrote that directly uses the WinHttpHandler but I want to code just against the HttpClientHandler. ``` private HttpClient CreateHttpClient() { var httpMessageHandler = new System.Net.Http.WinHttpHandler(); httpMessageHandler.SendTimeout = TimeSpan.FromSeconds(200); httpMessageHandler.ReceiveDataTimeout = TimeSpan.FromSeconds(200); httpMessageHandler.ReceiveHeadersTimeout = TimeSpan.FromSeconds(200); var httpClient = new HttpClient(httpMessageHandler); httpClient.Timeout = TimeSpan.FromSeconds(200); return httpClient; } ``` 2862 area-System.Collections Add HashSet ctors with capacity Replaces PR #2122. I squashed and fixed up @MarkusSintonen's commits from that PR. Fixes #382. New tests are not included due to System.Collections.dll being a partial facade, and the repo not currently having the infrastructure to enable adding tests for new surface area in partial facades. cc: @terrajobst, @KrzysztofCwalina 2865 area-System.Threading Leaky abstraction in ThreadPool.[Unsafe]RegisterWaitForSingleObject I just went into the pitfall of using a waithandle in two different registrations. The documentation states the following: > The wait thread uses the Win32 WaitForMultipleObjects function to monitor registered wait operations. Therefore, if you must use the same native operating system handle in multiple calls to RegisterWaitForSingleObject, you must duplicate the handle using the Win32 DuplicateHandle function. It seems quite bad when the documentation is encouraging you to write non-portable code, and P/Invoking to DuplicateHandle requires full trust. Would it be a big problem to detect that the same handle is there twice with two different registrations and correctly wait for it and queue both action on the thread pool when the waithandle becomes ready? 2869 area-Meta Merge future branch back into master This issue is just for tracking the changes we have merged into future so when we merge it back to master we know which API changes we need to modify contracts and ensure are supported on our other platforms like the desktop framework. At this point it isn't clear when this will happen but we want to keep track of this item. --- Merged PRs into future --- - [x] #1384 Regex group should provide a Name property. - [x] #1783 Add ConcurrentDictionary GetOrAdd/AddOrUpdate overloads with generic arg - [x] #1756 Implement generic interfaces on Regex collections - [x] #2436 Add async XLinq document/element loading and saving - [x] #2862 Add HashSet ctors with capacity 2873 area-System.IO Unix NamedServerPipeStream disposal no longer deletes the FIFO Per the discussion in #2764, the clean up of a Unix FIFO named pipe upon disposal of the NamedPipeServerStream that created it has been removed. - Resolves #2764 cc: @stephentoub 2875 area-System.IO Removed the Unix restriction on MaxNumberOfServerInstances There were previously checks to ensure that MaxNumberOfServerInstances was within the same boundaries as it was on Windows despite never actually being used in the Unix implementation. This commit removes those checks and adds platform specific tests for the new behavior. - Resolves #2764 2877 area-System.Collections Propose Adding a GetOrAdd(TKey key, Func valueFactory) to Dictionary "On `ConcurrentDictionary` there is the very useful function `TValue GetOrAdd(TKey key, Func valueFactory)`. The pattern that the method represents is one that issued commonly on the ""normal"" `Dictionary` as well. I propose adding such a method for this common use case. Besides simplifying it will also make switching between the two dictionaries simpler. " 2882 area-System.Security Same naming classes appear in multiple nuget packages. I know you guys are busy building the .net core packages. At the moment, I face a challenge about the following errors when target dnxcore50 Error CS0433 The type 'HashAlgorithm' exists in both 'System.Security.Cryptography.Hashing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' and 'System.Security.Cryptography.Primitives, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' Error CS0433 The type 'HMACSHA1' exists in both 'System.Security.Cryptography.Algorithms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' and 'System.Security.Cryptography.Hashing.Algorithms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' Error CS0433 The type 'RandomNumberGenerator' exists in both 'System.Security.Cryptography.Algorithms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' and 'System.Security.Cryptography.RandomNumberGenerator, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' Previously, we solve this kind of issues by add alias in project. Since we have new project file which is project.json. I have run out of idea how to add alias. Can one of masters give me some hints on this? 2883 area-System.Net Removed DumpHeaders dependency & coded to avoid unnecessary splitting ``` modified: ../../System.Net.Http/src/System/Net/Http/Unix/CurlHandler.cs ``` This fixed the useragent bug. The issue was DumpHeaders adding the same header twice in case of muti values header. 2885 area-System.Security X509 ExportMultiplePrivateKeys test failed in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/3248/console ``` Discovering: System.Security.Cryptography.X509Certificates.Tests Discovered: System.Security.Cryptography.X509Certificates.Tests Starting: System.Security.Cryptography.X509Certificates.Tests System.Security.Cryptography.X509Certificates.Tests.CollectionTests.ExportMultiplePrivateKeys [FAIL] System.Security.Cryptography.CryptographicException : Error occurred during a cryptographic operation. Stack Trace: d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\Internal\Cryptography\Pal.Windows\StorePal.Export.cs(82,0): at Internal.Cryptography.Pal.StorePal.Export(X509ContentType contentType, String password) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\X509Certificate2Collection.cs(123,0): at System.Security.Cryptography.X509Certificates.X509Certificate2Collection.Export(X509ContentType contentType, String password) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\X509Certificate2Collection.cs(116,0): at System.Security.Cryptography.X509Certificates.X509Certificate2Collection.Export(X509ContentType contentType) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Security.Cryptography.X509Certificates\tests\CollectionTests.cs(482,0): at System.Security.Cryptography.X509Certificates.Tests.CollectionTests.ExportMultiplePrivateKeys() Finished: System.Security.Cryptography.X509Certificates.Tests ``` 2886 area-System.Reflection Reflection types should be made extensible "I'm creating a subclass of FieldInfo and there's an error in the constructor saying ""'ConstructorInfo' does not contain a constructor that takes 0 arguments"". The reason for this is that my constructor is calling the default constructor of FieldInfo, which seems to be unavailable. That's because FieldInfo's constructor was changed to internal. As there is no other constructor defined in FieldInfo it's now impossible to extend it. It's the same for ConstructorInfo, EventInfo, MethodInfo, PropertyInfo, TypeInfo and Type. Is this intended? " 2890 area-System.Net Porting System.Net.NameResolution. "Porting System.Net.NameResolution and tests. Adding a PAL layer for this contract together with x-plat implementations. Pending work is marked either as ""Shims"" or with TODO comments. Feature #893 " 2891 area-System.Net Tracking items for System.Net.NameResolution This issue is tracking TODOs and Shims removal for PR #2890 : 1. Contract changes to System.Net.Primitives: 1. [ ] Add public ctor for `SocketException(SocketError errorCode, int platformError)` 2. [ ] `IPAddress` - allow access to internal buffer via internal contract or new public API on `System.Net.Primitives`. 3. [ ] `SocketAddress[int]` - allow direct access to internal buffer via internal contract or new public API in `System.Net.Primitives`. 2. Add UnitTests required by changes made during refactoring: 1. [ ] `NameResolutionPal.GetHostByAddr(IPAddress)` : test with BIGENDIAN 2894 area-System.Net Tracking x-plat issues for System.Net.NameResolution Feature #2484 : X-Plat comments from PR #2890 related to System.Net.NameResolution: 1. Update errno helpers. 2. Add Unit/Pal test to validate potential indexing issues found in NameResolutionPal.Unix.cs CreateHostEntry(Interop.libc.hostent\* hostent); Given the design error, I suggest similar tests for the entire file. 3. `TryGetNameInfo`'s allocation of `hostname`: consider using stackalloc / StringBuilderCache. 4. Intermittent PAL test failure: `System.Net.NameResolution.PalTests.NameResolutionPalTests.GetHostByName_HostName_GetHostByAddr` MESSAGE: Assert.Equal() Failure Expected: 1XY.XYZ.XYZ Actual: 255.2.0.0 +++++++++++++++++++ STACK TRACE: at System.Net.NameResolution.PalTests.NameResolutionPalTests.GetHostByName_HostName_GetHostByAddr() 1. Intermittent PAL test failure: `System.Net.NameResolution.PalTests.NameResolutionPalTests.TryGetAddrInfo_HostName_TryGetNameInfo` MESSAGE: Assert.Equal() Failure Expected: Success Actual: HostNotFound +++++++++++++++++++ STACK TRACE: at System.Net.NameResolution.PalTests.NameResolutionPalTests.TryGetAddrInfo_HostName_TryGetNameInfo() 2895 area-System.IO 5 FileSystem tests are failing on OS X 5 test in System.IO.FileSystem are failing on OS X System.IO.FileSystem.Tests.FileInfo_GetSetTimes.SettingUpdatesProperties [FAIL] Assert.All() Failure: 2 out of 6 items in the collection did not pass. [1]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000Z Actual: 2015-08-19T23:25:34.0000000Z at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.FileInfo_GetSetTimes.<>c__DisplayClass3_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) [0]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000-08:00 Actual: 2015-08-19T16:25:34.0000000-07:00 at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.FileInfo_GetSetTimes.<>c__DisplayClass3_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) Stack Trace: at System.IO.FileSystem.Tests.FileInfo_GetSetTimes.SettingUpdatesProperties() System.IO.FileSystem.Tests.DirectoryInfo_GetSetTimes.SettingUpdatesProperties [FAIL] Assert.All() Failure: 2 out of 6 items in the collection did not pass. [1]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000Z Actual: 2015-08-19T23:25:34.0000000Z at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.DirectoryInfo_GetSetTimes.<>c__DisplayClass3_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) [0]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000-08:00 Actual: 2015-08-19T16:25:34.0000000-07:00 at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.DirectoryInfo_GetSetTimes.<>c__DisplayClass3_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) Stack Trace: at System.IO.FileSystem.Tests.DirectoryInfo_GetSetTimes.SettingUpdatesProperties() System.IO.FileSystem.Tests.DirectoryInfo_ToString.ParentDirectory [FAIL] Assert.Equal() Failure ↓ (pos 0) Expected: Actual: / ↑ (pos 0) Stack Trace: at System.IO.FileSystem.Tests.DirectoryInfo_ToString.ParentDirectory() System.IO.FileSystem.Tests.Directory_GetSetTimes.SettingUpdatesProperties [FAIL] Assert.All() Failure: 2 out of 6 items in the collection did not pass. [1]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000Z Actual: 2015-08-19T23:25:39.0000000Z at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.Directory_GetSetTimes.<>c__DisplayClass5_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) [0]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000-08:00 Actual: 2015-08-19T16:25:39.0000000-07:00 at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.Directory_GetSetTimes.<>c__DisplayClass5_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) Stack Trace: at System.IO.FileSystem.Tests.Directory_GetSetTimes.SettingUpdatesProperties() System.IO.FileSystem.Tests.File_GetSetTimes.SettingUpdatesProperties [FAIL] Assert.All() Failure: 2 out of 6 items in the collection did not pass. [1]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000Z Actual: 2015-08-19T23:25:40.0000000Z at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.File_GetSetTimes.<>c__DisplayClass5_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) [0]: Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 2014-12-01T12:00:00.0000000-08:00 Actual: 2015-08-19T16:25:40.0000000-07:00 at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) at System.IO.FileSystem.Tests.File_GetSetTimes.<>c__DisplayClass5_0.b__0(Tuple`3 tuple) at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) Stack Trace: at System.IO.FileSystem.Tests.File_GetSetTimes.SettingUpdatesProperties() 2897 area-System.Net "WinHttpRequestStream.Flush Appends ""0\r\n\r\n"" in Chunked Mode" "WinHttpRequestStream.Flush writes ""0\r\n\r\n"" to the stream every time the method is called in when _chunkedMode is true. This behavior seems wrong. The _endChunk should only be added to the end. " 2898 area-Meta Change tests to use consistent namespaces "Namespace used in test projects are all over the place: - default (previous version of System.Text.Encoding.CodePages tests) - Test (PLINQ) - $packageNamespace.Tests - $packageNamespace.UnitTests .... and potentially more. Is there any guidance for what we should be using? The only potentially applicable guideline would be ""Follow the style of the current project"", which doesn't help much in the case of new work (or say what any canonicalization tool would change things to). " 2903 area-System.Net Removed DumpHeaders dependency & coded to avoid unnecessary splitting ``` modified: System/Net/Http/Unix/CurlHandler.cs ``` I messed up while updating the changes in remote branch. Tried to fix that but could not. Will figure out what caused the issue. Sending new PR with updated changes. Copying the comments here. Sorry for any inconvenience. 2904 area-System.IO Shims should use intptr_t for file descriptors Once we get all of the IO stuff shimmed, it would be better to use intptr_t instead of int32_t for file descriptors as then we can marshal SafeHandles to descriptors without the current workarounds. 2905 area-System.Net Merge changes from TFS 2906 area-System.IO Added ZLib decompression support to DeflateStream The DeflateStream class currently uses ZLib for compression (if available) but doesn't use ZLib for decompression. This commit adds support to use either ZLib or the Managed implementation for decompression (inflation). - Resolves #2024; Added ZLib support in InflaterZLib.cs - Inflater.cs the old managed implementation is renamed to InflaterManaged.cs - Extracted interface of necessary Inflater methods into the IInflater interface from which InflaterManaged and InflaterZLib inherit. - Roughly a 3X speedup for decompression when using the ZLib library instead of the Managed implementation! 2907 area-System.Net Merge changes from TFS 2910 area-System.Security new X509Certificate2(byte[]) should return the signing cert for PKCS#7 on Unix The Windows code supports reading a PKCS#7 signed or PKCS#7 signed-and-enveloped structure; but in single certificate mode it doesn't return certs[0], it returns the certificate which signed the structure. My test files (produced by Windows (certmgr.msc and X509Certificate2Collection::Export)) aren't signed, so Windows emits an exception (`new CryptographicException(ErrorCode.CRYPT_E_SIGNER_NOT_FOUND)`). Without a sample to see what's going on here, it's hard to make compatible behavior. So, for now, the Unix implementation will just throw, even if it could have worked. 2913 area-System.Security Add support for exporting PKCS7 files on Unix 2914 area-System.Threading System.Threading.Tasks.Tests.CancelWait.TaskCancelWaitTestCases.TaskCancelWait1 failed in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst_prtest/1989/testReport/junit/System.Threading.Tasks.Tests.CancelWait/TaskCancelWaitTestCases/TaskCancelWait1/ MESSAGE: Expected Result to lie between 1.63 and 1.65 for completed task. Actual Result -42. Using n=Light IsCanceled=False Expected: True Actual: False +++++++++++++++++++ STACK TRACE: at System.Threading.Tasks.Tests.CancelWait.TaskCancelWaitTest.VerifyResult(TaskInfo current) at System.Threading.Tasks.Tests.CancelWait.TaskCancelWaitTest.b__3(TaskInfo current) at System.Threading.Tasks.Tests.CancelWait.TaskInfo.Traversal(Action`1 predicate) at System.Threading.Tasks.Tests.CancelWait.TaskCancelWaitTest.Verify() at System.Threading.Tasks.Tests.CancelWait.TaskCancelWaitTest.RealRun() at System.Threading.Tasks.Tests.CancelWait.TaskCancelWaitTestCases.TaskCancelWait1() 2915 area-System.Net HttpClient xplat: Fix bugs in polling and callback logic - On OSX, the read end of the pipe sometimes wakes up the poll with a POLLOUT. Since we are interested in only reading on it, scoping down to POLLIN prevents this from happening. Else we attempt a read on it which fails - CheckForCompletedTransfers was being unnecessarily called from the socket callback. We are only interested in MSG_DONE after the socket callback indicates removal. This will anyway wake up PollFunction which calls CheckForCompletedTransfers. This race is also causing a seg fault sometimes because curl_multi_socket_action has been called on an fd that was getting cleaned up via curl_easy_cleanup (call sequence is CurlSocketCallback --> CheckForCompletedTransfers --> EndRequest) - Also removed some unnecessary CharSet attributes cc: @stephentoub @davidsh @CIPop @SidharthNabar @pgavlin 2916 area-System.Net Introducing a bunch of tests for http client handler This checkin tests the client handler in following scenarios: - Http and Https calls - Authentication - Redirection - Redirection + Authentication - custom headers - cookies 2928 area-System.Net Add ALPN support to System.Net.Security.SslStream Porting to newest bug tracker. https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/6264363-add-support-for-alpn-to-system-net-security-sslstr https://connect.microsoft.com/VisualStudio/feedback/details/812003/system-net-security-sslstream-needs-support-for-alpn-to-negotiate-spdy-http2-connections Microsoft drove the HTTP/2 working group to adopt the ALPN TLS Extension for use in negotiating HTTP/2 connections. It succeeded. SChannel got ALPN support in the Windows 8 timeframe. .NET's SslStream class does not have ALPN support, preventing low-level socket clients (like Fiddler) from sending ALPN tokens to servers during TLS handshaking, in turn blocking support of the HTTP/2 protocol. The code change to SslStream is probably a dozen lines at worst. The current workaround for those blocked by this shortcoming is to take on a **massive** security risk and ship either BouncyCastle or OpenSSL (or re-wrap SChannel, I guess). 2930 area-System.Security Change Windows GetRSAPrivateKey to prefer CNG When GetRSAPrivateKey was added into the repository the CNG types weren't yet online. Now that they are, we should have GetRSAPrivateKey and GetRSAPublicKey use CNG when able, and fallback to CAPI for legacy support. 2934 area-System.Net Allow setting IP protection level on Socket - P2P is not possible without it It is not possible for peers to establish a connection with eachother behind NAT(s) using Windows sockets without setting the IP socket option IP_PROTECTION_LEVEL. Currently there is no way to do this on a `DatagramSocket` or .NET Core `System.Net.Socket.Socket`, thus there is _no way make Peer 2 Peer Windows Store Apps_. The issue on [Microsoft Connect](https://connect.microsoft.com/VisualStudio/feedback/details/1431526/nat-traversal-udp-hole-punching-not-possible-in-windows-store-apps-even-with-internet-client-server-capability). In .NET Framework one used [`System.Net.Sockets.IPProtectionLevel`](http://referencesource.microsoft.com/#System/net/System/Net/Sockets/Socket.cs,5eb8a772f197999a). We need to add this facility `Windows.Networking.Sockets.DatagramSocket` and/or `System.Net.Sockets.Socket` 2936 area-System.Net .NET Core's System.Net.Sockets.Socket Send and Receive methods use old async technique There is no documentation on the `System.Net.Sockets.Socket` provided in .NET Core (beyond the XML code documentation) but it seems to provide a subset of .NET Framework desktop [`Socket`s](https://msdn.microsoft.com/en-us/library/system.net.sockets.socket.sendasync%28v=vs.110%29.aspx) members. The only methods to send and receive are [`SendAsync`](https://msdn.microsoft.com/en-us/library/system.net.sockets.socket.sendasync%28v=vs.110%29.aspx) and [`RecieveAsync`](https://msdn.microsoft.com/en-us/library/system.net.sockets.socket.receiveasync%28v=vs.110%29.aspx). Despite the names they are not `async` methods but an old school variant to which one supplies the `SocketAsyncEvenArgs` parameter _shudder_. We need to bring .NET Core's `Socket` up-to-date and provide `async` variants. Also consider synchronous methods since `Socket`s are often used in in background processes with dedicated threads where that is wanted (applications tend to use higher application level type such as `HttpClient`). 2938 area-System.Runtime GC tests GetTotalMemoryTest_ForceCollection, GetGenerationTest test failed in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_release/1057/console ``` GCTests.GetTotalMemoryTest_ForceCollection [FAIL] Assert.True() Failure Expected: True Actual: False Stack Trace: d:\j\workspace\dotnet_corefx_windows_release\src\System.Runtime\tests\System\GC.cs(295,0): at GCTests.GetTotalMemoryTest_ForceCollection() ``` We should probably just delete these tests. 2942 area-System.Threading Request to Add Timer.Dispose(WaitHandle) API in CoreCLR. We are working on porting our .Net 4.5 application to the .Net Core CLR platform. We have multiple services implemented in our application which need to do some work every so often. Currently this is implemented using System.Threading.Timer with a callback method that is responsible for scheduling the next callback. During cleanup, we want to wait until we are sure that the callback is no longer running and will not run in future. Currently we do this using Threading.Timer.Dispose(WaitHandle) with additional synchronization mechanism to avoid the ObjectDisposedException. Since CoreCLR doesn't have Timer.Dispose(WaitHandle) implemented, every application/service now has to implement Timer.Dispose(WaitHandle) logic using some synchronization mechanism in their code. This adds complexity at each place where this API is used currently. We would like to carry forward the same behavior and so would request to add this API [Timer.Dispose(WaitHandle)] back in CoreCLR. Thanks, Mehul. 2944 area-System.Net Fix an assert in CurlHandler multi handle release path - If a timeout happens, the socket fd is not removed from _fdSet. When SafeCurlMultiHandle is finalized, it checks if _fdSet is empty and this assertion was failing. So ensuring that all codepaths of EndRequest result in cleanup from _fdSet cc: @stephentoub @davidsh @CIPop @SidharthNabar @pgavlin 2953 area-System.Security Add RSA.Create() to utilize the correct platform-dependent RSA implementation 2958 area-System.IO Add coverage to DriveInfo tests FileSystem.DriveInfo.dll coverage raised to at or near testable maximum. Added several scenarios that weren't present before and also split up Volume Label setting/getting to several tests so that errors can be better diagnosed. 2961 area-System.IO Avoid unnecessary allocations when using FileStream "This was originally a PR (https://github.com/dotnet/coreclr/pull/1429), turned into an issue as a result of the comments there. The idea is to avoid 4KB allocation for buffer whenever we need to work with large number of files. Consider the following code: ``` foreach (var file in System.IO.Directory.GetFiles(dirToCheck, ""*.dat"")) { using (var fs = new FileStream(file, FileMode.Open)) { // do something to read from the stream } } ``` The problem is that each instance of FileStream will allocate an independent buffer. If we are reading 10,000 files, that will result in 40MB(!) being allocated, even if we are very careful about allocations in general. See also: https://github.com/dotnet/corefx/pull/2929 The major problem is that FileStream will allocate its own buffer(s) and provide no way to really manage that. Creating large number of FileStream, or doing a big writes using WriteAsync will allocate a lot of temporary buffers, and generate a _lot_ of GC pressure. As I see it, there are a few options here: - Add a constructor that will take an external buffer to use. This will be the sole buffer that will be used, and if a bigger buffer is required, it will throw, instead of allocating a new buffer. - Add a pool of buffers that will be used. Something like the following code: [ThreadStatic] private static Stack[] _buffersBySize; private static GetBuffer(int requestedSize) { if(_buffersBySize == null) _buffersBySize = new Stack[32]; ``` var actualSize = PowerOfTwo(requestedSize); var pos = MostSignificantBit(actualSize); if(_buffersBySize[pos] == null) _buffersBySize[pos] = new Stack(); if(_buffersBySize[pos].Count == 0) return new byte[actualSize]; return _buffersBySize[pos].Pop(); ``` } private static void ReturnBuffer(byte[] buffer) { var actualSize = PowerOfTwo(buffer.Length); if(actualSize != buffer.Length) return; // can't put a buffer of strange size here (prbably an error) ``` if(_buffersBySize == null) _buffersBySize = new Stack[32]; var pos = MostSignificantBit(actualSize); if(_buffersBySize[pos] == null) _buffersBySize[pos] = new Stack(); _buffersBySize[pos].Push(buffer); ``` } The idea here is that each thread has its own set of buffers, and we'll take the buffers from there. The Dispose method will return them to the thread buffer. Note that there is no requirement to use the same thread for creation / disposal. (Although to be fair, we'll probably need to handle a case where a disposal thread is used and all streams are disposed on it). The benefit here is that this isn't going to impact the external API, while adding the external buffer will result in external API being visible. " 2963 area-System.Net Handle leak with WinHttpHandler "The following test demonstrates an unbounded leak of both memory and handles: ``` C# [Fact] public void TestLeak() { while (true) { HttpClient client = new HttpClient(); client.GetStringAsync(""http://www.httpbin.org"").Wait(); // client.Dispose(); } } ``` At least one problem appears to be that without explicitly Dispose'ing of the HttpClient instance, the underlying session SafeHandle is never cleaned up. When the session handle is created, it's DangerousAddRef'd (https://github.com/dotnet/corefx/blob/master/src/System.Net.Http.WinHttpHandler/src/System/Net/Http/WinHttpHandler.cs#L992), and then DangerousRelease is only called if explicitly Dispose'ing (https://github.com/dotnet/corefx/blob/master/src/System.Net.Http.WinHttpHandler/src/System/Net/Http/WinHttpHandler.cs#L555). " 2965 area-System.Net Dns.GetHostAddressesAsync and HttpClient can't run on Linux System.AggregateException: One or more errors occurred. ---> System.ArgumentException: The AddressFamily Unspecified is not valid for the System.Net.IPEndPoint end point, use InterNetwork instead. Parameter name: socketAddress at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult) at System.Net.Dns.EndGetHostAddresses(IAsyncResult asyncResult) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() at ConsoleApp1.Program.d__2.MoveNext() --- End of inner exception stack trace --- at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at ConsoleApp1.Program.Main(String[] args) 2967 area-System.Net Add System.Net.NameResolution to README.md Add System.Net.NameResolution to README.md. 2972 area-System.Net Add more tests for WebHeaderCollection Excluded from #2380 2973 area-System.Net Fix HttpClientHandler timeouts Since HttpClientHandler doesn't expose all the timeout used by the underlying WinHttpHandler, we need to disable those timeouts in favor of the HttpClient.Timeout property. Addresses issue #2857. 2982 area-System.Net WebSocket extensions From using the WebSocket class, I think these extensions make sense: ``` C# Task ReceiveTextAsync(CancellationToken cancellationToken = default(CancellationToken)); Task> ReceiveBinaryAsync(CancellationToken cancellationToken = default(CancellationToken)); Task SendAsync(string message, CancellationToken cancellationToken = default(CancellationToken)) Task SendAsync(ArraySegment message, CancellationToken cancellationToken = default(CancellationToken)) ``` - All methods have a defaulted CancellationToken. - The Receive method implementations read until EndOfMessage. - The Receive methods send back a Close message when receiving one. The message sent back echos the status code which was received. - The Receive methods throw when the MessageType is different than that of the method signature. (For most protocols, the user will know what MessageType he expects to come next.) - The string overloads handle the encoding/decoding. 2984 area-System.Collections Installing System.Collections.Immutable on .NET 4.5 brings unnecessary packages "When System.Collections.Immutable nuget package is installed to project targeting .NET 4.5, some unnecessary dependencies are brought in: ``` ``` NuGet is apparently smart enough to not add those packages as references as they are only meant for .NET Core, but I still feel they should not even be added to packages.config and downloaded. [System.Collections.Immutable nuget package](https://www.nuget.org/packages/System.Collections.Immutable/) only specifies dependencies for .NET Core (DotNET 5.0). I believe that adding empty dependencies group for .NET 4.5 as can be seen in [AutoMapper nuget package](https://www.nuget.org/packages/AutoMapper/) (for example) would fix the issue and those dependencies would not be downloaded. Have not adding other dependency groups been done on purpose or is it an oversight? Thanks! " 2988 area-System.Net How to implement System.Net.NameResolution on FreeBSD? The assembly `System.Net.NameResolution` does not compile on FreeBSD. Looking at the sources it seems to have corresponding source files implementing it on Linux and OS X. Looks like a good candidate for a shim, to unify the implementation. Especially given that the sockets API is pretty consistent across POSIX systems. @nguerrera do you plan a `System.Net.Native` shim to cover this? \cc @josteink 2992 area-System.Net Merge changes from TFS 2996 area-System.Globalization System.Globalization: Expose DateTimeFormat GetShortestDayName to .NET Core contract I found what looks like a mistake in the .NET Core contracts for DateTimeFormat. This class exposes two sets of APIs: some methods, and their corresponding properties. For example, DateTimeFormat has a GetMonthName(int) method, that returns the name of the specified month. And then it has a corresponding property MonthNames, that is a string array of all the month names. This is true for Month Names, Abbreviated Month Names, Day Names, and Abbreviated Day Names. And in the full framework, this holds true for Shortest Day Names: string GetShortestDayName(DayOfWeek) & string[] ShortestDayNames. However, in the .NET Core contracts, the string[] property ShortestDayNames is public, but the method GetShortestDayName(DayOfWeek) is not exposed. I see this [is true for Silverlight](https://msdn.microsoft.com/en-us/library/system.globalization.datetimeformatinfo%28v=vs.95%29.aspx) as well. We should expose this method in the .NET Core. 3000 area-System.Net Switch to using async WinHTTP APIs in WinHttpHandler The current version of WinHttpHandler calls into WinHTTP using a synchronous pattern. We need to switch to using the async pattern instead, in order to provide the required performance and scalability for server scenarios of ASP.NET 5. 3002 area-System.Net Add System.Net.Http.Native shim and seed it with curl initialization cc @sokket @bartonjs @stephentoub 3007 area-System.Linq Calls with out enum arguments fail when expression tree is interpreted. "``` C# public enum MyEnum { Value } public class Foo { public static void Bar(out MyEnum o) { o = MyEnum.Value; } public void Test() { var x = Expression.Variable(typeof (MyEnum), ""x""); var expression = Expression.Lambda( Expression.Block( new [] { x }, Expression.Call(null, typeof(Foo).GetMethod(""Bar""), x))); expression.Compile()(); } } ``` Interpreter throws - $exception {""Object of type 'System.Int32' cannot be converted to type 'App1.MyEnum&'.""} System.ArgumentException " 3008 area-System.Xml Short-circuit key comparison on equal indexes in EnumerableSorter in System.Xml.XDocument System.Xml.XDocument has its own copy of EnumerableSorter and possibly can benefit from the same fix as in https://github.com/dotnet/corefx/pull/2977 3009 area-System.IO PathHelper stackalloc version can't grow Will always fail with MAX_PATH if it goes beyond when resolving paths (relative, short names, etc.). Need to change to allow switching between stack and heap when needed. 3014 area-System.Reflection MSDN doc wrong about virtual CustomAttributeData.AttributeType property in .NET 4.5 According to MSDN, [System.Reflection.CustomAttributeData.AttributeType](https://msdn.microsoft.com/en-us/library/system.reflection.customattributedata.attributetype%28v=vs.110%29.aspx) is virtual on .NET4.5/4.6. It isn't however in reality. I suppose it's confused because in CoreFx the property _is_ indeed [virtual](https://github.com/dotnet/corefx-progress/blob/67c6fd93d8b24f5f82490e295950214a45769f4d/src-full/System.Reflection.cs#L54). Can we get this fixed? 3015 area-System.Net HttpClientHandler on unix (CurlHandler) hangs forever in case of negative scenarios "When curl handler encounters exception in its execution, it hangs forever. ``` using (var handler = new HttpClientHandler()) { handler.MaxAutomaticRedirections = 5; var client = new HttpClient(handler); await client.GetAsync(""http://httpbin.org/redirect/6""); } ``` " 3016 area-System.Net HttpClientHandler on unix (CurlHandler) Cookie Behavior different between Unix and Windows There are subtle differences in the way HttpClientHandler behaves w.r.t cookies. On windows if we set httpHandler.UseCookies = true but if the user does not provide a valid cookieContainer, we get InvalidOperationException. In case of CurlHandler, we are ignoring it. 3028 area-System.Security Add X509Chain tests for expiration boundary conditions The X509Chain object supports an explicit time for the override. Using a statically built chain, no revocation mode, and an explicit time, it should be possible to get a very deterministic test for the chain assessment. 3033 area-System.Net Fix error handling in WinHttpHandler StartRequest Addresses dnu restore exception 2547 https://github.com/aspnet/dnx/issues/2547/ - Fix a code path where `requestHandle` could be `null`. - Add unit test to simulate this error code path. 3034 area-System.Security Add support for OCSP on Unix OCSP = Online Certifciate Status Protocol 3035 area-Microsoft.CSharp System.Dynamic.Runtime tests should probably move to under Microsoft.CSharp What we have currently under System.Dynamic.Runtime\tests are mostly tests for C# dynamic. While they do provide coverage for System.Dynamic, they target various language features and only indirectly target DLR machinery. This is confusing. (see, for example, discussion in https://github.com/dotnet/corefx/pull/2941) Tests that target and require C# dynamic should go to under Microsoft.CSharp. If there are any tests that directly targets DLR and do not require C# dynamic, they can remain under System.Dynamic.Runtime. 3037 area-System.Diagnostics TestProcessStartTime test is failing on OS X The TestProcessStartTime test in System.Diagnostic.Process.ProcessTests class is failing with the following error: System.Diagnostics.ProcessTests.ProcessTests.TestProcessStartTime [FAIL] Assert+WrapperXunitException : File path: Y:\Repositories\personal\dotnet\corefx\src\System.Diagnostics.Process\tests\ProcessTests.cs. Line: 387 ---- Assert.InRange() Failure Range: (635766571991261530 - 635766571991772530) Actual: 635766571650984930 Stack Trace: at Assert.WrapException(Exception e, String callerFilePath, Int32 callerLineNumber) at Assert.InRange[T](T actual, T low, T high, String path, Int32 line) at System.Diagnostics.ProcessTests.ProcessTests.TestProcessStartTime() ----- Inner Stack Trace ----- at Assert.InRange[T](T actual, T low, T high, String path, Int32 line) 3043 area-System.Linq Allow empty BlockExpression expressions Issue #1877 discusses having `SwitchExpression` objects with no cases. One might similarly have a case where `BlockExpression` elements were being created dynamically and one hit the case of their being no expressions within the block. Such a block would be essentially the same as `Expression.Empty`, but arrived at through happening to hit zero item in a dynamic construction. It would seem that the arguments [here](https://github.com/dotnet/corefx/pull/2803#issuecomment-136514021) about zero-case switches apply equally to zero-item blocks. 3045 area-System.Net WebException.Status on Unix returning wrong result for DNS failures ``` Discovering: System.Net.Requests.Tests Discovered: System.Net.Requests.Tests Starting: System.Net.Requests.Tests System.Net.Requests.Test.HttpWebRequestTest.GetResponseAsync_ServerNameNotInDns_ThrowsWebException [FAIL] Assert.Equal() Failure Expected: NameResolutionFailure Actual: UnknownError Stack Trace: at System.Net.Requests.Test.HttpWebRequestTest.GetResponseAsync_ServerNameNotInDns_ThrowsWebException() Finished: System.Net.Requests.Tests ``` 3046 area-System.Linq Use BlockExpression subclasses more widely. BlockExpression has several subclasses that cover various numbers of sub- expressions when the type is the same as the last expression, and there are no variables. These are only used in one or two of the possible paths under which they make sense. Use them more widely. 3048 area-System.Reflection Merge branch 'master' of https://github.com/dotnet/corefx into dev/metadata 3050 area-System.Net Refactor CurlHandler for threading and resource cleanup "@davidsh, @CIPop, @pgavlin: The first commit in this PR enables the System.Net.Requests tests to run on Unix in CI. Just a few minor tweaks. @kapilash: The second commit is your commit with new tests from #2916, which this PR replaces. @kapilash, @vijaykota: The third commit is the majority of the changes for this PR... CurlHandler is now split into three types: - CurlHandler, which represents the shell of the handler, contains some helper functions, and contains a ""MultiAgent"". It submits ""EasyRequest""s to the ""MultiAgent"". - CurlHandler.EasyRequest, which represents all of the state associated with a SendAsync request on the HttpClient. - CurlHandler.MultiAgent, which represents the owner of a libcurl multi handle and all of the processing associated with it. The MultiAgent is responsible for actually processing EasyRequests, which CurlHandler queues to it. MultiAgent is designed to not require any cleanup. When the first EasyRequest arrives, it spins up a worker that creates a multi handle to process the request. Any additional requests to arrive while that worker is active will be processed by that worker and multi handle. When no work remains, the multi handle and the worker goes away. If/when another request arrives, a new multi handle / worker are spun up. Etc. Almost all of the locking previously there was removed, as the worker thread is now solely responsible for doing all work on the multi handle; any requests for processing come in through the requests queue processed by that thread. This design avoids threading issues we faced with regards to accessing state managed by the multi handle without holding the lock (e.g. polling file descriptors) and avoids having multiple threads blocked waiting on the lock to do processing associated with the multi handle. It also makes resource cleanup much easier and more deterministic: all state associated with the easy handle is cleaned up when that thread is done processing the easy handle, and all state associated with the multi handle is cleaned up when the worker thread shuts down. The change also ensures that we always complete the response message, and that any exceptions that occur are propagated as needed to Read on the response stream (fixes #3015). Further, the design will make it trivial in the future to change the relationship between a CurlHandler and a MultiAgent: - If we want one MultiAgent shared across all CurlHandlers, we just change the _agent field to be static instead of instance. - If we want a pool of MultiAgents, we can simply create a collection of them and round-robin between them from one or more CurlHandlers. - Etc. All of the configuration code remains, e.g. what options are set on the easy and multi handles, but the code has been refactored. I also include CURLOPT_NOSIGNAL to avoid multithreading problems per the docs. Along the way, I cleaned up some stale interop that's gotten left behind through various iterations, and I fixed a few small issues related to behavioral differences from Windows, e.g. CookieContainer never returning null, UseCookies defaulting to true, etc. (Fixes #3016) " 3051 area-Meta Create a C++ coding standard for the PAL layer Create a C++ code style document describing coding standards and naming standards to use the PAL layer. Also, fix up the current PAL code to adhere to the coding standard 3054 area-System.Runtime Add AssemblyLoadContext.LoadUnmanagedDllFromPath helper API "## Scenario Overrides of `System.Runtime.Loader.AssemblyLoadContext.LoadUnmanagedDll` that just wish to provide alternative paths are forced to write platform-specific code to call `LoadLibrary` or `dlopen` to load the library and return the library handle. For example: ``` c# internal class CustomAssemblyLoadContext : AssemblyLoadContext { [DllImport(""kernel32"")] private static extern IntPtr LoadLibrary(string path); [DllImport(""libdl"")] private static extern IntPtr dlopen(string path, int flags); protected override IntPtr LoadUnmanagedDll(String unmanagedDllName) { if (!ShouldLoadFromCurrentDirectory(unmanagedDllName)) { return IntPtr.Zero; // use the default loader } var libraryPath = Path.Combine(Environment.CurrentDirectory, unmanagedDllName); IntPtr libraryHandle = EnvironmentHelper.IsWindows ? LoadLibrary(libraryPath) : dlopen(libraryPath, 0); if (libraryHandle == IntPtr.Zero) { // ... (more error handling) throw new DllNotFoundException(unmanagedDllName); } return libraryHandle; } } ``` The proposal is to add an API to `AssemblyLoadContext` to provide a platform-independent way of loading an unmanaged library: ``` c# internal class CustomAssemblyLoadContext : AssemblyLoadContext { protected override IntPtr LoadUnmanagedDll(String unmanagedDllName) { if (!ShouldLoadFromCurrentDirectory(unmanagedDllName)) { return IntPtr.Zero; // use the default loader } var libraryPath = Path.Combine(Environment.CurrentDirectory, unmanagedDllName); return LoadUnmanagedDllFromPath(libraryPath); } } ``` ## Proposed API ``` c# namespace System.Runtime.Loader { public abstract class AssemblyLoadContext { // Relevant existing members protected abstract Assembly Load(AssemblyName assemblyName); protected Assembly LoadFromAssemblyPath(string assemblyPath); protected virtual IntPtr LoadUnmanagedDll(String unmanagedDllName); // Proposed members protected IntPtr LoadUnmanagedDllFromPath(String unmanagedDllPath); } } ``` ## Details ### Usage and behavior - `unmanagedDllPath` must be an absolute path to the library to load - No changes are made to the specified search path or library name in `unmanagedDllPath` before attempting the load. For example, no prepending ""lib"" and no appending "".so"". However, the system may make changes, for instance, LoadLibrary on Windows appends a "".dll"" if an extension is not specified. - `LoadUnmanagedDllFromPath` would try to load the library using default load flags, and upon success, would return the library handle (value returned by LoadLibrary/dlopen) - `LoadUnmanagedDllFromPath` would throw an exception upon failure to load the library - Upon a successful load, callers from LoadUnmanagedDll overrides are expected to return the library handle from LoadUnmanagedDll, since there is no equivalent of FreeLibrary/dlclose - Upon returning a nonzero value from LoadUnmanagedDll, the loader would attempt to load the entry point from the library ### Exceptions - `ArgumentNullException` - Self-explanatory - `ArgumentException` - `unmanagedDllPath` is an empty string or a relative path - The following exceptions are inherited by the default loader implementation: - `BadImageFormatException` - Windows only - Failed to load the library - `DllNotFoundException` - Did not find the library or failed to load the library - Note that the default loader may also throw `EntryPointNotFoundException`. Since the entry point is loaded upon returning from `LoadUnmanagedDll`, a successful return from `LoadUnmanagedDll` may still be followed by this exception. " 3055 area-System.Net System.Uri fails on long non ascii domains "This domain in russian: ``` csharp Console.WriteLine(new Uri(@""http://высококачественные-пиломатериалы.рф"", UriKind.Absolute)); output: http:/// ``` I start truncate domain from end: ``` csharp Console.WriteLine(new Uri(@""http://высококачественные-пиломатериал.рф"", UriKind.Absolute)); // still bad output: http:/// Console.WriteLine(new Uri(@""http://высококачественные-пиломатериа.рф"", UriKind.Absolute)); // remove one more char, ok output: http://высококачественные-пиломатериа.рф/ ``` `Uri.TryCreate` with schema `//` fails with `Index was outside the bounds of the array.`: ``` csharp Uri uri; Uri.TryCreate(@""//высококачественные-пиломатериалы.рф"", UriKind.Absolute, out uri); exception: Index was outside the bounds of the array. ``` " 3056 area-System.Security MD5.Create() fails on Linux/Mac (was: Windows interop getting called on Unix systems) When calling `Internal.NativeCrypto.Cng.Interop.BCryptOpenAlgorithmProvider` (e.g. when using `MD5.Create()`) on Unix systems the Windows Interop seems to be called. On both Ubuntu 14.04 and OSX I get `System.DllNotFoundException: Unable to load DLL 'BCrypt.dll'`. 3061 area-Serialization Clean up unused directives in serialization code 3062 area-System.Security CryptographicException(int hr) should assign Message based on the code Using Interop.mincore.GetMessage (or whatever the correct API is) on Windows, and the OpenSSL ERR_ APIs on Unix. 3063 area-System.Security Calls to X509_STORE_add_crl should check for unexpected errors `0x0B07D065` (X509_R_CERT_ALREADY_IN_HASH_TABLE) is expected, anything else should throw. Blocked on #3062, because currently we can either get the numeric code in the exception, or the string, but not both. 3065 area-Infrastructure Add coreclr<-> corefx repo integration build We should have an outerloop coreclr<->corefx integration build. 3066 area-System.Numerics Some Vector tests overflow their test values on AVX2 Some of the tests for Vector inadvertently overflow some of the array data they generate, because they do some arithmetic on the index of each element. For example: ``` CSharp T[] values2 = new T[Vector.Count]; for (int g = 0; g < Vector.Count; g++) { values2[g] = (T)(dynamic)(g * 5); } Vector vec2 = new Vector(values2); ``` On AVX2, there are 32 bytes in a Vector. 32 \* 5 = 160, which has far overflowed a signed byte. Anything over 25 elements here will overflow, which is breaking the tests, as they are doing GreaterThan/LessThan/etc. comparisons between the three arrays generated. Actually, the (unsigned) Byte versions of these tests will also break once they run on AVX512, since 64-byte vectors will easily overflow an unsigned byte with this calculation, as well. These should be rewritten to be more robust to this sort of issue. Note: this issue doesn't show up on the builds here; we don't run the tests in a configuration that would hit this (AVX2, x64, Release) 3067 area-System.Net Implement ReadAsync on CurlHandler's response stream 3068 area-System.Net Implement ReadAsync on CurlHandler response stream Today, CurlHandler provides a response content stream that doesn't override ReadAsync. As such, any consumers reading data using ReadAsync will actually end up blocking a thread pool thread doing a synchronous Read. This commit fixes that by implementing ReadAsync to be truly asynchronous with no additional threads blocked. If data is already available, a task is returned immediately. Otherwise, the reader asynchronously waits for the writer to publish data and then consumes that data. The overall design of the response stream remains the same as it was. However, there are some notable changes: - A SemaphoreSlim is now used instead of a ManualResetEventSlim for the _writerPublishedDataEvent. This was done because SemaphoreSlim supports both synchronous and asynchronous waiting. - The _readerRequestingDataEvent was removed, as it was no longer necessary. - We no longer allocate an UnmanagedMemoryStream per write operation, instead simply storing the IntPtr buffer and long length on the response stream. - Additional reorganization was done, primarily to share logic between Read and ReadAsync. - A bunch of comments and debug asserts were added. A second commit enables the asynchronous wait to be canceled by either the internal shutdown CancellationToken or by the CancellationToken provided by the caller to ReadAsync. Fixes #3067 cc: @vijaykota, @kapilash, @davidsh, @pgavlin, @CIPop 3069 area-System.Net Delete stale HttpClientHandler.Unix.cs file from repo There are currently two HttpClientHandler.Unix.cs files checked in: - One in System.Net.Http\src\System\Net\Http - One in System.Net.Http\src\System\Net\Http\Unix The former is the original stubbed out version, throws from every property, and isn't referenced by the .csproj, whereas the latter is where all of the recent development related to CurlHandler has happened. This PR simply deletes the former to avoid confusion, and then moves the latter up one level so that it continues to be next to its .Windows.cs counterpart. (The GitHub diff makes it looks like I actually changed .cs file content, which I didn't; I left it as two commits to make it more evident that I simply deleted a file, moved another, and then made a two-line change to the .csproj.) cc: @davidsh, @vijaykota, @kapilash 3071 area-System.Net Fix for #3045. Fixing WebException.Status on Unix This commit fixes #3045. WebExecption class currently looks at the value of underlying HResult to set the value of the Status. We need to set the value of HttpRequestMessage.HResult to appropriate CURLcode in case of errors. WebException maps this to the relevant WebExceptionStatus 3072 area-System.Security System.Security.Cryptography.Csp should support all runtimes "I want to use RSACryptoServiceProvider, but System.Cryptography.Csp seams to be not available when targeting ""dotnet"". I looked around and it doesn't look like it's moved to another assembly. Are there any plans about this? " 3077 area-System.Security WindowsPrincipalTests.WindowsPrincipalIsInRoleNeg test fails locally I get this failure when trying to run the System.Security.Principal.Windows tests locally: ``` Discovering: System.Security.Principal.Windows.Tests Discovered: System.Security.Principal.Windows.Tests Starting: System.Security.Principal.Windows.Tests WindowsPrincipalTests.WindowsPrincipalIsInRoleNeg [FAIL] System.Exception : The trust relationship between this workstation and the primary domain failed Stack Trace: \corefx\src\System.Security.Principal.Windows\src\System\Security\Principal\NTAccount.cs(343,0): at System.Security.Princ ipal.NTAccount.TranslateToSids(IdentityReferenceCollection sourceAccounts, Boolean& someFailed) \corefx\src\System.Security.Principal.Windows\src\System\Security\Principal\NTAccount.cs(204,0): at System.Security.Princ ipal.NTAccount.Translate(IdentityReferenceCollection sourceAccounts, Type targetType, Boolean& someFailed) \corefx\src\System.Security.Principal.Windows\src\System\Security\Principal\NTAccount.cs(174,0): at System.Security.Princ ipal.NTAccount.Translate(IdentityReferenceCollection sourceAccounts, Type targetType, Boolean forceSuccess) \corefx\src\System.Security.Principal.Windows\src\System\Security\Principal\WindowsPrincipal.cs(69,0): at System.Security .Principal.WindowsPrincipal.IsInRole(String role) \corefx\src\System.Security.Principal.Windows\tests\WindowsPrincipalTests.cs(14,0): at WindowsPrincipalTests.WindowsPrinc ipalIsInRoleNeg() Finished: System.Security.Principal.Windows.Tests ``` cc: @pallavit 3085 area-System.Net Improve SafeWinHttpHandle reference counting logic to prevent leaks "This PR mainly addresses ""handle leaks"" #2963, #2762, #2793. In investigating the leaks, I discovered that our use of the SafeHandle pattern was not quite right. In addition, SafeHandle has some rather quirky lifetime management aspects with respect to the Dispose() and DangerousRelease() methods. After reviewing the source code for SafeHandle itself, I discovered a better pattern to using. Native Windows WinHTTP handles wrapped by SafeHandle need additional lifetime management protection. I also needed to add finalizers to the WinHttpHandler, WinHttpRequestStream and WinHttpResponseStream classes. Finally, I added unit tests to validate our behavior assumptions and to track leaks in the WinHttpHandler logic. This change does not affect the SafeWinHttpHandleWithCallback class that is used by System.Net.WebSocket.Client. But that class is incorrectly using the SafeHandle pattern w.r.t. Dispose() and I will be fixing that in a later PR. " 3089 area-System.Net CurlHandler blocking write callback can lead to deadlock With the current implementation of CurlHandler in HttpClient on Unix, when downloaded data is available, libcurl invokes a callback to provide that data to the application. Until a reader is available to consume all of that data, the callback blocks. I'd previously noted this as a performance problem to be fixed, but it's actually a much worse problem in that it can easily lead to deadlock. Here's a new test to demonstrate: ``` C# [Fact] public async Task GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock() { using (var client = new HttpClient()) { const int NumGets = 5; Task[] responseTasks = Enumerable.Range(0, NumGets) .Select(_ => client.GetAsync(HttpTestServers.RemoteGetServer, HttpCompletionOption.ResponseHeadersRead)) .ToArray(); for (int i = responseTasks.Length - 1; i >= 0; i--) { using (HttpResponseMessage response = await responseTasks[i]) { await AssertSuccessfulGetResponse(response, HttpTestServers.RemoteGetServer, _output); } } } } ``` Here we kick off N downloads and then iteratively process each to completion before moving on to process the next. This means that we only try to read from a single download at a time, and if this doesn't happen to be the same download that has data available next, our reader will block waiting for data, which will never come because the libcurl callback is blocked waiting for a reader for a different download. We should fix this by employing libcurl's ability to pause connections. When the write callback is invoked, if there isn't currently a read pending on that download, the write callback should return to libcurl to inform it that the connection needs to be paused. When the reader later comes along, it needs to unpause that connection. Obviously there will be some synchronization required here, but it shouldn't be too bad. cc: @vijaykota, @kapilash 3091 area-System.Net Enabling Testcases for POST/PUT methods 3093 area-System.Linq Add static method to create an IEnumerable with one element I find that I need the following static method quite often: ``` csharp public static class MoreEnumerableMethods { // ... public static IEnumerable Singleton(TResult element) { return Repeat(element, 1); } } ``` This seems like a really good fit for `Enumerable` itself, where it would live alongside `Repeat`, `Empty`, and `Range`. I want to be able to say `Enumerable.Singleton(x)`, not `MoreEnumerableMethods.Singleton(x)`. Even Eric Lippert has [complained](http://ericlippert.com/2013/03/18/monads-part-eight/) about the lack of this method: > I wish there was a static method on Enumerable specifically for making a one-element sequence. I have the code (and tests) ready on [a branch in my fork](https://github.com/dotnet/corefx/compare/master...benjamin-hodgson:singleton#diff-95a14d3b4ae14339b623dd95c7537fd0R1497). I can open a pull request right away, subject to any required API reviews. I'm happy to discuss things like the method name (perhaps `One` or `Singular` would be a better choice?). 3096 area-System.Net Improve WinHttpHandler and SafeWinHttpHandle reference counting logic "This PR mainly addresses ""handle leaks"" #2963, #2762, #2793. In investigating the leaks, I discovered that our use of the SafeHandle pattern was not quite right. Native Windows WinHTTP handles wrapped by SafeHandle need additional lifetime management protection to preserve parent-child relationships. Closing the parent handle too early will invalidate the child handle. So, added a SetParentHandle() method to SafeWinHttpHandle to protect against this. This also simplified the passing of handles to the WinHttpResponseStream class. I added finalizers to the WinHttpHandler and WinHttpResponseStream classes. Note that the WinHttpRequestStream class doesn't need a finalizer because its lifetime is short within the processing of the SendAsync() method. Finally, I added unit tests to validate the behavior assumptions and to track leaks in the WinHttpHandler logic. This change does not affect the SafeWinHttpHandleWithCallback class that is used by System.Net.WebSocket.Client. But that class is incorrectly using the SafeHandle pattern w.r.t. Dispose() and I will be fixing that in a later PR. " 3098 area-System.Reflection Fix System.Reflection.Metadata.nuspec dependency 3103 area-System.Numerics System.Numerics.Vectors NuGet package does not work with F# "The [System.Numerics.Vectors](https://www.nuget.org/packages/System.Numerics.Vectors/) package (v4.1.0) doesn't seem to work correctly with F#: Whenever I try to use it, it installs correctly, but the compiler always refers to the corresponding .NET 4.6 reference assembly (""C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework.NETFramework\v4.6\System.Numerics.Vectors.dll"", in my case) instead of the downloaded assembly in the packages directory. When installing the NuGet package into a C# project, it found that a `` item was added to the csproj file, redirecting the reference to the proper assembly (e.g. `..\packages\System.Numerics.Vectors.4.1.0\lib\net46\System.Numerics.Vectors.dll`). For F# projects however this `` is _not_ added to the fsproj file. Adding it manually fixes the problem. " 3105 area-System.Net Avoid blocking CurlHandler write callback When receiving the body of an HTTP response, libcurl provides the data received via a callback that's invoked synchronously from the thread pumping libcurl's processing. This callback can't return until it's handed off all of the available data, and thus the current implementation of CurlHandler blocks that thread until a reader has consumed the data. This is both expensive and dangerous. It's expensive because we end up blocking a thread until some reader comes along, and it's dangerous because this can easily lead to deadlock. A libcurl multi handle may be used by only one thread at a time, which means that if the handle is being used by a thread while blocked on the write callback, no other thread can access the multi handle, which means no other callbacks for other concurrent requests can be invoked. If multiple operations were queued, and then the data for them arrived in an order that didn't exactly match the order of readers consuming the data, deadlock would ensue. This change makes it so that we no longer block the callback, taking advantage of a pausing mechanism libcurl provides that allows one to pause a connection until it's needed. I rewrote the CurlResponseStream to manage this. Now when the callback has data to write, it attempts to transfer it to the stream. If there's a reader already waiting for the data (represented by a task stored on the stream), the callback simply transfers its data to the reader. If the callback has too much data than would fit in the reader, the remaining data is buffered on the stream. If instead the callback finds that either there isn't a waiting task or that there's already some remaining data buffered from a previous callback, it instead pauses the connection, such that no more callbacks will be invoked until we unpause. From the reader side, a reader who tries to read data from the stream can pull data from the remaining buffer if there is any, or can register a task with the stream that will be filled with data at some point when callback has data available. As part of debugging some issues with this change, I added a verbose tracing mechanism that's enabled by compiling the library with the CURLHANDLER_VERBOSE constant (the same that we use for enabling libcurl's verbose diagnostic output). I left the tracing in as I found it valuable in diagnosing issues with the library, but we can remove it later if desired. I also added a test that deadlocks before the change and that completes successfully after. cc: @vijaykota, @kapilash Fixes #3089 3106 area-System.Linq Intersect not optimized for HashSet Hello, I found some old blog about an optimization of Intersect when the underlying collection is a HashSet. I have been caught of this issue in a contest and using the custom implementation made a huge performance difference. Here the blog by Patrick Smacchia with a full description of the issue : http://www.codebetter.com/patricksmacchia/2011/06/16/linq-intersect-2-7x-faster-with-hashset/ Patrice 3107 area-Meta [Feature] General purpose data binding support? "Please forgive me if this is the wrong place to post this; but it certainly seems like a good place :) I think that it would be extremely useful if there were a general-purpose data binding mechanism providing a consistent way to synchronize the properties of objects making it for developers to have consistent support for patterns that benefit from data binding such as the popular MVVM and MVPVM design patterns. Also including the `IValueConverter` interface and related functionality that is included in the `System.Windows.Data` namespace of the fully featured .NET framework but perhaps in a less platform-specific namespace like `System.Data`? Whilst not applicable in the following example; perhaps a general purpose XAML implementation would also be of benefit perhaps allowing a XAML representation of a custom selection of UI controls outside the scope of Windows or Xamarin forms? I see this being of particular use in .NET Core based game engines. # Example Use Case (User Code) For example, if the Unity game engine was using .NET Core with such a feature it would then be quite straightforward to add binding support to the Unity UI controls simply by having them implement `INotifyPropertyChanged`. ## Programmatic Bindings Continuing with the Unity example; some `MonoView`-derived component could be added to the root of each composition of UI controls allowing for manual binding similar to this: ``` // ExampleView.cs public class ExampleView : MonoView { [SerializeField] private InputField _firstNameInputField; ... protected override void OnBind() { Bindings.Add(new Binding(_firstNameInputField, ""text"", ViewModel, ""FirstName"", BindingMode.TwoWay)); } } ``` Where the base classes look like this: ``` // MonoView.cs public abstract class MonoView : MonoBehaviour { public MonoView() { Bindings = new BindingCollection(); } protected BindingCollection Bindings { get; private set; } ... protected virtual void OnBind() { } protected virtual void OnUnbind() { // Properly disposes all bindings (unsubscribes from source/target objects). foreach (var binding in Bindings) { binding.Dispose(); } Bindings.Clear(); } } // MonoView{T}.cs public abstract class MonoView : MonoView where TViewModel : class { ... } ``` ## Designer Specified Bindings In addition to programmatic binding (like above) it would also be fairly straightforward to create a custom ""PropertyBinder"" `MonoBehaviour` component which could be added to UI controls to allow designers to make similar associations using just the Unity ""Inspector"" window. ``` +---------------------------------------------------------------------------+ | Property Bindings Component: | | +-------------------------------------------------[ Remove Binding ]--+ | | | ___________________________________________ | | | | Control Property: | text | \/ | | | | | ------------------------------------------- | | | | ___________________________________________ | | | | Data Context Property: | FirstName | \/ | | | | | ------------------------------------------- | | | +---------------------------------------------------------------------+ | | _________________________ | | | Add Another Binding | | | ------------------------- | +---------------------------------------------------------------------------+ ``` ## Bindings properly disposed when removed/cleared from the `Bindings` collection When the view is unbound from the data context all `Bindings` would be properly disposed allowing the view to be reused and bound to another view model. # Closing Words I'm sure that you folks have probably already got something like this planned; but I wanted to post this feature request just in case it isn't on the roadmap :) " 3113 area-System.Net Porting System.Net.Security Missing issues for TODO comments. Will create the tracking issue and update the code, author (dotnetbot) and reference the tracking issue. @davidsh @bartonjs @pgavlin @stephentoub @SidharthNabar PTAL. 3114 area-System.Net Tracking items for System.Net.Security This issue is tracking TODOs and Shims removal for PR #3113: - [ ] Replace Interop.Secur32 names with OS names from the SDK (sspi.h and schannel.h) - [X] Replace the X509Certificate ctor shim as soon as it is available. - [X] Move _SSPIWrapper.cs OS specific classes to Interop.SSPI - [ ] Remove StreamAsyncHelper references and implement FixedSizeReader and SslState using TPL instead of APM. - [ ] _SslState.cs:1299: Within FinishWrite() the callback is called under `lock (this)`. We need to investigate if there is any chance of a deadlock. - [X] Remove SSLv2 support from SslStream. - [ ] Add new Cypher and Hashing algorithms to enums within the contract. - [ ] Split _SecuritySafeHandles.Windows.cs - [ ] In NativeSSPI: Unit/Functional test required to understand the intent and pin behavior between Desktop and CoreFX. > Decrypt checked qop != SECQOP_WRAP_NO_ENCRYPT. Sign used it. Verify just ignores it? - [ ] Replace all usage of ArrayList, Hashtable with appropriate modern collections. - [ ] Behavior impacting: check client certs for the client EKU within `private bool AcquireClientCredentials(ref byte[] thumbPrint)` - [ ] Replace the implementation of SpnDictionary with `Dictionary<>` or `ConcurrentDictionary<>`. - [X] Provide TPL Stream overrides for SslStream. (By #5541.) - [ ] Investigate ways to bring back the NetworkStream specific optimizations (`BeginMultipleWrite`, etc) - [ ] Move public parameter checks within their respective public methods. (See comments in #5541) - [ ] In `CompleteAuthToken` and `ApplyControlToken`: remove the extra allocation of a `CredHandle`; refactor the code to avoid duplication; remove the `AddRef` if the `refContext` was previously `null`. (See #11489) 3124 area-System.Net Content-Type getting set incorrectly in CurlHandler "This test passes on Windows but fails on Unix: ``` C# [Theory, MemberData(""PostServers"")] public async Task PostAsync_CallMethod_StreamContent(Uri remoteServer) { using (var handler = new HttpClientHandler()) using (var client = new HttpClient(handler)) { byte[] data = new byte[1234]; new Random(42).NextBytes(data); HttpContent content = new StreamContent(new MemoryStream(data)); HttpResponseMessage response = await client.PostAsync(remoteServer, content); Assert.Equal(HttpStatusCode.OK, response.StatusCode); string responseContent = await response.Content.ReadAsStringAsync(); Assert.Contains(Convert.ToBase64String(data), responseContent); } } ``` It looks like the Content-Type header is being set inappropriately. " 3126 area-System.Collections Circular buffer data structure As far as I know, there is no circular buffer implementation in .NET (https://en.wikipedia.org/wiki/Circular_buffer). Some simple implementations over an array or a Queue are discussed here, for instance: https://social.msdn.microsoft.com/forums/vstudio/en-US/416a2175-b05d-43b1-b99a-a01c56550dbe/circular-buffer-in-net 3128 area-Infrastructure Microsoft.CSharp fails to build when running MSBuild on Mono When using build.sh on Linux Microsoft.CSharp runs into some build issues. I am going to disable it for now and follow up with this issue. 3131 area-System.Net Fix for #3124. CurlHandler sets invalid content-type for POST requests libcurl defaults to a Content-Type:application/x-www-form-urlencoded for Post operations. This default behavior is inconsistent with dotnet API. Current PR fixes this issue by overriding the default value. 3133 area-System.Xml Investigate/Remove custom internal test frameworks There are a number of projects with their own internal test frameworks, that need to be investigated, and potentially removed in favor of xunit. To be clear, I'm not talking about projects with helper objects and methods (although some should be re-evaluated now that we're using xunit), I'm thinking of stuff like System.Xml.ReaderWriter.Tests (and its multiple constituent projects): ![image](https://cloud.githubusercontent.com/assets/5569508/9761804/e5ab2cb6-5739-11e5-8e2b-510d780c0e29.png) ...and if you'll notice, xunit isn't reporting the skipped tests, because it doesn't see them. Are those important? I really doubt reporting is turning those up. And hopefully fails percolate.... Some of these frameworks are pretty heavily entrenched in their host projects, and dismantling them would be a large-scale effort. The ones I'm aware of (can recall) at the moment are: - System.Collections - System.Threading.Tasks - System.Xml.ReaderWriter ...although there are undoubtedly more. 3134 area-Infrastructure System.ServiceProcess.ServiceController.Tests needs admin privleges... ...but the tests don't check for permission ahead of time/don't report that as a possible error. 3135 area-System.Diagnostics Support an optional exception handler in TelemetryListener/TelemetrySource "I've been playing around with the new `TelemetryListener`/`TelemetrySource` classes and have found them to be a great fit for a simple eventing pattern. However an unexpected impact on firing `WriteTelemetry` was that if a subscriber fails, no subsequent subscribers will have their `OnNext` method called. From reading the various comments - particularly @vancem's https://github.com/dotnet/corefx/pull/2932#issuecomment-134328582 - on the PR I understand that this stems from how RX approaches exceptions, yet in the case of telemetry and multi-cast events it seems like there will be many cases where one broken subscriber shouldn't bring the system down. Would it be possible to consider an overload to `WriteTelemetry` which looks like this: ``` c# public override void WriteTelemetry(string telemetryName, object arguments, Func exHandler) { for (TelemetrySubscription curSubscription = _subscriptions; curSubscription != null; curSubscription = curSubscription.Next) { try { curSubscription.Observer.OnNext(new KeyValuePair(telemetryName, arguments)); } catch (Exception ex) { if (!exHandler(ex)) { break; } } } } ``` This would then support calls that would allow the sender to pass a function that can evaluate for each exception if the calls should continue: ``` c# source.WriteTelemetry(""Send"", o, ex => !(ex is CatastrophicException) ); ``` Alternatively an approach could be considered that uses `AggregateException` that simply collects all exceptions and throws them collectively at the end. " 3136 area-System.Net TransferEncodingChunked not being set by CurlHandler for streams of unknown length "Exact same test as in #3124, but with a non-seekable stream instead of a seekable one, such that HttpClient can't access the stream's length: ``` C# [Theory, MemberData(""PostServers"")] public async Task PostAsync_CallMethod_UnknownLengthStreamContent(Uri remoteServer) { var handler = new HttpClientHandler(); using (var client = new HttpClient(handler)) { byte[] data = new byte[1234]; new Random(42).NextBytes(data); HttpContent content = new StreamContent(new NonSeekableMemoryStream(data)); HttpResponseMessage response = await client.PostAsync(remoteServer, content); Assert.Equal(HttpStatusCode.OK, response.StatusCode); string responseContent = await response.Content.ReadAsStringAsync(); Assert.Contains(Convert.ToBase64String(data), responseContent); } } private sealed class NonSeekableMemoryStream : MemoryStream { public NonSeekableMemoryStream(byte[] data) : base(data) { } public override bool CanSeek { get { return false; } } } ``` TransferEncodingChunked should be set to true in this case, but CurlHandler isn't doing so, such that the test passes on Windows but fails on Unix. " 3137 area-System.Net CurlHandler synchronously reads from request stream, blocking libcurl processing If the request message has a request content stream to send, libcurl retrieves that data via read callbacks done during its processing. These read callbacks are currently Read'ing from the content stream, which blocks the processing while the read is being completed. If the content stream is a memory stream, this is fine, as such reads will complete immediately. And even for some other stream types, like FileStream, the reads should complete quickly enough that there's no problem. But for streams that take a longer time, this can cause significant delays in CurlHandler's ability to process other pending HttpClient requests. It can also lead to deadlock if the request stream for one request is being fed by the response stream of another request being processed by the same handler. 3138 area-System.Net Avoid blocking CurlHandler read callback PR #3105 addressed the issue we were facing with libcurl and the write callback used when libcurl has response body data available. We were blocking libcurl's callback until a reader was available to consume the data, resulting in both significant scalability issues as well as the potential for easy deadlock. This was fixed by making the whole operation async via libcurl's ability to pause connections. The same issue exists for libcurl's read callback, used by libcurl to read data from the request stream to send as part of the request. The issue here isn't as impactful as on the write side, because a) a common case has no request stream or has a MemoryStream that serves up its data immediately, and b) other streams (e.g. FileStream) typically serve their data quickly even if blocking is involved and don't interact with the HttpClient in a way that could lead to deadlock. However, it's still an issue, and it's still possible to construct cases where deadlock is possible, e.g. if the output stream of one request is used as the input stream of another request associated with the same libcurl multi handle. This commit addresses the read callback case by relying on the same pause mechanism used for the write callback. When libcurl requests data, if the source is a memory stream, we just do the read and hand back the data. Otherwise, we issue a ReadAsync on the stream. If the operation completes immediately, we synchronously hand the data back to libcurl, storing any extra beyond what libcurl asked for. If the operation didn't complete immediately, we store it for later, and pause the connection, requesting an unpause when the read operation completes. Subsequent read requests from libcurl first try to be satisfied from any previous ReadAsync with data remaining. As part of this, I also removed the pause state tracking I'd previously added. In my previous PR as part of adding support for pausing, I added a field to EasyRequest to attempt to mirror the current paused state as known by libcurl. This was done to try to avoid requesting unpausing in cases where we'd like to be unpaused but have already requested it. However, this appears to have been premature optimization, as the case is relatively rare, the overheads relatively low for processing the extra request, and it causes additional headaches once we're able to pause sending/receiving individually. I added several tests for different kinds of streams. This required fixing #3136, which was preventing any data from being sent from non-seekable request streams. Fixes #3136 Fixes #3137 cc: @vijaykota, @kapilash, @davidsh, @pgavlin, @CIPop 3140 area-Meta Turn on CheckForOverflowUnderflow on all projects in Debug Right now, only System.Reflection.Metadata has CheckForOverflowUnderflow on in Debug. I think there's value in doing this across the board. Expressions that are allowed to overflow by design should be explicitly marked `unchecked`. cc @weshaggard @stephentoub 3141 area-System.Linq Expression interpreter - AddOvf does not throw OverflowException In AddInstruction.cs, the following code `(Int16)checked((Int16)l + (Int16)r)` seems to be wrong and should be `checked((Int16)((Int16)l + (Int16)r))` instead. Repro: `Expression.Lambda( Expression.AddChecked( Expression.Constant(Int16.MinValue), Expression.Constant(Int16.MinValue) ) ).Compile().DynamicInvoke()` throws TargetInvocationException with an OverflowException when using the compiler. When using the interpreter, it succeeds. 3144 area-System.Net Remove references to TaskAPMExtensions in System.Net tests Remove Common\tests\System.Net\TaskAPMExtensions.cs and rewrite tests using TPL instead. 3147 area-System.Linq Expression interpreter - NewArrayBounds does not throw OverflowException Consider the following fragment: `Expression.Lambda(Expression.NewArrayBounds(typeof(int), Expression.Constant(0), Expression.Constant(-1))).Compile().DynamicInvoke()` With the expression compiler, this throws OverflowException. With the expression interpreter, the call to Array.CreateInstance results in an ArgumentOutOfRangeException. 3149 area-System.Linq Expression compiler - Inconsistency with language specification Section 7.9 of the C# language specification describes the number of bits to shift. It looks like this rule is not honored by the expression compiler which omits the `and` instruction to perform the mask, prior to performing the `shl` or `shr` instruction. The following repros the discrepancy: Expression<Func<long?, int?, long?>> e = (x, y) => x << y; Func<long?, int?, long?> f = (x, y) => x << y; Func<long?, int?, long?> g = e.Compile(); var c = f(-9223372036854775808, -2147483648); var d = g(-9223372036854775808, -2147483648); Console.WriteLine(c); Console.WriteLine(d); Note that the implementation of the expression interpreter relies on C# compiled code fragments for << and >>, so this will cause a different result for .Compile() between frameworks that use the expression compiler versus those that use the interpreter. 3150 area-Infrastructure Cannot build System.Collections.Concurrent.Tests in Visual Studio 2015 I get the following error: ``` xunit.abstractions 2.0.0 is not compatible with DNXCore,Version=v5.0. ``` Any suggestions or work-around? 3151 area-System.Net CurlHandler supports only ClientCertificateOption.Manual Today CurlHandler supports only clientCertificateOption.Manual. Now with the availability of cross platform X509Store, CurlHandler must support ClientCertificateOption.Automatic as well. 3153 area-System.Net Could CurlHandler support UNIX sockets? There is support in `libcurl` for using UNIX sockets instead of TCP (http://curl.haxx.se/libcurl/c/CURLOPT_UNIX_SOCKET_PATH.html). In a microservice world, this capability is very useful for communication between discrete services on a single server, reducing the overhead of TCP sockets and improving performance. As a precedent, the Kestrel HTTP server now supports UNIX sockets, which Nginx can consume when reverse-proxying (see https://github.com/aspnet/KestrelHttpServer/pull/157). I took a quick look to see if I could implement this, but there doesn't seem to be a point where a socket file path could be added in, so I opted for starting a discussion instead. :open_mouth: 3156 area-System.IO Can we determine if a FileSystem is case sensitive or not through an API? I'm exploring the possibility of making our cross-platform file globbing to determine if it acts as case sensitive or not based on current file system. It can't be determined just by the current operating system since case insensitive file system can still be mounted on Linux/Unix/Darwin. Is there an API can help to determine if a file system at given path is case sensitive. 3163 area-System.Net Replace TRAVE to TRACE_VERBOSE throughout the code. **Workitem:** 1. Ensure that all instances of `TRAVE` are replaced by `TRACE_VERBOSE` throughout the code base. 2. Ensure that no other name is used to enable verbose tracing. (If others exist, they should be replaced by `TRACE_VERBOSE`.) --- **Initial Design Discussion**: In System.Net code we have debug-only verbose tracing guarded by a variable named `TRAVE` (TRAce VErbose). The following have been brought up during PRs for this code: 1. Standardize a single such name across corefx 2. Change the name (seems like a typo) Please add your suggestions and then convert this to a code issue to track changes across the entire repo. 3172 area-Meta Standardize flags declarations in native shims We have a mix of raw integers and casting scoped enums right now. See https://github.com/dotnet/corefx/pull/3142#issuecomment-139330591: A general comment on raw integers vs. enum's on C++ side for flags. I've been doing the latter based on my recollection that storing a value that is not any one of the enum members is unspecified or undefined in standard C++. However, I'm no longer sure after looking at recent edits on cppreference.com: http://en.cppreference.com/mwiki/index.php?title=cpp/language/static_cast&diff=78871&oldid=74481 My understanding was on the LHS of that diff. I don't know yet if the edits reflect errata on the site matching my own misunderstanding or just updates tracking the next version of the standard. Even if it is well defined, it requires a lot of ugly casting, which kind of eliminates the benefit of strong typing. It's not apparent in this change because your flags come in from C# with the right values, but once you have to twiddle the bits, e.g. ConvertOpenFlags, it gets messy. Assuming it is in fact well defined, we could have a macro to define bitwise operators on what we deem Flags enums. This would serve double-duty as documentation to C# caller to add [Flags]. I'm fine leaving this for a later change, but this is something we should standardize one way or the other. 3178 area-System.Net Redirect handling for cookies and creds This fix realigns CurlHandler with WinHttpHandler w.r.t Redirect handling. When redirect is detected, we clear the credentials if the credentials were originally provided as NetworkCredentials Otherwise we use the forwarding URI to get the new set of credentials. Like wise, in case of redirects, we get the cookies again from the cookieContainer In addition, this commit also collects the cookies from the response header 3179 area-System.Runtime Enhance Guid type with implicit type conversions to useful primitive types "Currently, Guid feels a little bit left out. While other built-in types have implicit type convertions, or support from Convert, Guid have it's own ""bastion"" of methods and converters, and is a source of confusion for new developers. Is there any particual reasons (except historical) that Guid should not be part of built-in types and have a implicit convertion (from bytes or string)? " 3180 area-Serialization XmlSerializationWriter not visible "I get the following error: DNXCore,Version=v5.0 error CS0234: The type or namespace name 'XmlSerializationWriter' does not exist in the namespace 'System.Xml.Serialization' (are you missing an assembly reference?) Program.cs: ``` using System; namespace XmlSerializationWriterTest { public class Program { public void Main(string[] args) { System.Xml.Serialization.XmlSerializationWriter writer = (System.Xml.Serialization.XmlSerializationWriter)(new Object()); } } } ``` project.json: ``` { ""dependencies"": { }, ""frameworks"": { ""dnxcore50"": { ""dependencies"": { ""System.Xml.XmlSerializer"": ""4.0.10"" } } } } ``` The code of XmlSerializationWriter is here: https://github.com/dotnet/corefx/blob/41e203011152581a6c65bb81ac44ec037140c1bb/src/System.Xml.XmlSerializer/src/System/Xml/Serialization/XmlSerializationWriter.cs " 3183 area-System.Data Use modern data structures and array APIs for SqlClient SqlClient in its current state has code that has been around since a long time. The usage of data structures like HashTable should be migrated to Dictionary There are some array operations which are being done manually and can be migrated to Array APIs 3184 area-System.Data Update Interop Code in SqlClient to follow guidelines Based on PR https://github.com/dotnet/corefx/pull/2986 address the interop related comments. 3185 area-System.Net Add System.Net.Http.CurlHandler HttpClient is implemented as a sort-of plug-in model, where you provide to it a “handler” that actually does the heavy lifting. There’s a default HttpClientHandler type, which on Windows is implemented around winhttp and on Unix is implemented around libcurl. HttpClientHandler only exposes core knobs that are expected to be common across all possible implementations, but the underlying implementations often provide more functionality. So on Windows, we also have System.Net.Http.WinHttpHandler (a class in its own separate assembly), which is the same handler used by default on Windows under HttpClientHandler, but which as a public type exposes additional knobs for developers to configure, e.g. timeouts on individual portions of a request/response rather than for the all-up operation. There are such knobs we’d like to be able to configure with libcurl, e.g. targeting a UNIX domain socket as a proxy, but for which we’d need to expose CurlHandler publically in order to provide the right additional surface area. 3186 area-System.IO Add System.IO.FileSystem.Unix There are a bunch of common Unix file system constructs that we have no public surface area in .NET to control or interact with. For example, Unix permissions are a very common need for users/devs to interact with, but we don’t provide any surface area for getting (stat) or setting (chmod) these. We don’t provide surface area for changing ownership (chown), for creating symlinks (symlink) or hard links (link), etc. Devs can P/Invoke to these, but it’d be nice to have surface area for it to avoid devs needing to do that, especially given that some of these are impractical to P/Invoke to in any common way across Unix systems, e.g. the struct used by stat changes in both layout and size based on which distro/version you’re targeting, which is one of the many reasons we’ve moved away from direct P/Invokes to libc and instead have a think native shim. As a comparison, Mono has a library called Mono.Posix that provides essentially 1:1 wrappers between directly callable C# functions and the underlying POSIX functions they wrap. 3187 area-System.Security Add System.Security.Principal.Unix We provide the Windows-only System.Security.Principal.Windows.dll on Windows, which brings with it WindowsIdentity and WindowsPrincipal. We don’t currently have corresponding types for UnixIdentity and UnixPrincipal. This would support getting the current user/group, setting the effective user/group (kind of like Windows impersonation), looking up information about the user, etc. 3188 area-System.IO Add System.IO.Signals.Unix Signals are a ubiquitous mechanism across Unix systems for interactions between processes and the kernel. The only API we currently have for sending any signal is Process.Kill(), which sends a SIGKILL signal, but to write robust Unix applications (and services) you likely want to be able to both send signals and handle specific signals. The former can be done by a dev using a P/Invoke, but the latter would really require libcoreclr support to do correctly, especially since the runtime itself already hooks certain signals (e.g. a SIGINT signal sent by ctrl-C, which we then subscribe to the runtime for in Console.CancelEventHandler). 3189 area-Meta Add System.Configuration.Unix There’s a lot of Unix configuration information that apps could benefit from being able to read (e.g. sysconf, pathconf, ulimit, etc.) and potentially modify. Potentially support for parsing/outputting Unix configuration files as well. 3190 area-System.Net RemoteCertificateValidationCallback for HttpWebRequest The current implementation of HttpWebRequest in .Net 4.6 lacks RemoteCertificateValidationCallback similar to that of SslStream class. This makes mutual authentication impossible, especially when client becomes aware of mutual authentication enforced by server only after SSL handshake has been initiated. Currently all certificate callbacks for HttpWebRequest is managed by ServicePointManager class, I request to include this feature in corefx releases. Please see [this question](http://stackoverflow.com/questions/9294433/how-to-detect-client-certificate-authentication-request-in-httpwebrequest) in SO. 3199 area-System.Net Reduce allocations when parsing response headers in WinHttpHandler "There are currently many unnecessary allocations when creating the response message in `WinHttpHandler`. The native `WinHttpQueryHeaders` function is called a few times to retrieve response information, such as the version string, reason phrase, and raw headers. Each call to `WinHttpQueryHeaders` currently results in 3 allocations (9 total for the 3 calls): 1. The `StringBuilder` buffer passed to the native function to be filled in. 2. The `char[]` inside the `StringBuilder`. 3. The `string` allocated when `StringBuilder.ToString()` is called. On platforms that need to do manual decompression (lower than Win 8.1), there's another 2-3 allocations to get the Content-Encoding header value. We can reduce the allocations by 1) modifying the p/invoke signature for `WinHttpQueryHeaders` to take a `char[]` instead of `StringBuilder` and 2) allocate a single `char[]` large enough to be reused across multiple calls to `WinHttpQueryHeaders`. With those changes, 9 allocations are down to 2: an allocation for the raw headers `char[]` and another for the reason phrase `string`. There are many common reason phrases, such as ""OK"" for a 200 status code, or ""Not Found"" for a 404 status code. There's already internal code in `HttpStatusDescription` for looking up a known reason phrase string based on the status code, and we can make use of this existing code to avoid the reason phrase string allocation if the reason phrase `char[]` array segment is equal to a known reason phrase. 9 allocations down to 1, for common reason phrases. When parsing the raw headers, the current code is using `string.Split` to split the raw headers by lines separated by `""\r\n""`. There are a bunch of unnecessary allocations associated with the use of `string.split`: internally `string.Split` allocates two arrays to keep track of the indexes and separator sizes, there's the overall array returned from `Split`, plus each string in the array representing each line. Each line then needs to be split further to get the header name and value for each line. The header value was also being trimmed with `string.Trim`, another potential allocation. Also, the first line of the raw headers is the status line, which is being allocated as a `string` even though this line is currently being skipped. All of these intermediate allocations can be avoided by parsing the `char[]` directly, and only allocating `string`s for the individual header names and values. And we can avoid `string` allocations for known header names and values, similar to the known reason phrase optimization. Notes: - `HttpKnownHeaderNames.TryGetHeaderName` is looking up all known header name constants, which include both request and response headers, while we only really care about response headers. My opinion is that this is OK. I think it's easier to maintain the lookup of known headers if it is just looking up all known headers, instead of trying to segregate which ones are only response headers. - `HttpKnownHeaderNames.TryGetHeaderName` is currently doing an ordinal lookup. Header names are case insensitive, so I'd be open to changing this to do an case insensitive lookup. I kept it ordinal so the string remains the exact same as the response string from WinHTTP. However, it looks like WinHTTP normalizes the casing of headers it knows about (e.g. WinHTTP returns `""Content-Length""` when the actual server response is `""content-length""`), so maybe switching to case insensitive lookup isn't a big deal. - The PR is broken up into multiple commits to make it easier to review. The earlier commits are mostly just adding helpers and tests, and moving some of the internal files to `Common` (e.g. `HttpStatusDescription`). I can squash some of the commits together, if desired. - There are still a bunch of allocations associated with how headers are enumerated and stored internally on each request/response in `HttpClient` that I'm looking into improving in a separate PR. " 3205 area-System.Net Corefx HttpClientHandler CookieContainer property deviates from Desktop During the PR for #3178 , it was discovered that the `HttpClientHandler` of corefx deviates from desktop .Net version w.r.t the setter for `CookieContainer` property. Setting a `null` value results in an exception in the desktop version. In case of corefx, it results in an error during `SendAsync`. 3206 area-Infrastructure How to run Unit tests on Windows run on 32-bit CoreCLR? Is it possible to run corefx tests on 32 bit CoreCLR? After building corefx, I opened a corefx solution and changed `platform target``to``x86`` but it doesn't work. 3207 area-System.Security PFX certificates are not well-ordered across implementations Similar to issues #2743 and #2745, the fix would be to not use the `PKCS12_parse` and `PKCS12_create` helper functions, but to get into the nitty-gritty with PKCS12 primitives. This manifests as both writing order and reading order, to be consistent across Windows, Linux, and OSX. 3213 area-System.Net IPAddressParsing.ParseIPv6_ScopeId_Success failing on OS X ``` Assert.Equal() Failure ↓ (pos 7) Expected: fe08::1%13542 Actual: fe08::1 ↑ (pos 7) +++++++++++++++++++ STACK TRACE: at System.Net.Primitives.Functional.Tests.IPAddressParsing.ParseIPv6_ScopeId_Success() ``` 3215 area-System.IO Two FileSystemWatcher tests failing on OS X ``` 09:22:39 CreatedTests.FileSystemWatcher_Created_WatcherDoesntFollowSymLinkToFile [FAIL] 09:22:39 Didn't observe a symlink created event within 30000ms 09:22:39 Expected: True 09:22:39 Actual: False 09:22:39 Stack Trace: 09:22:39 at Utility.ExpectEvent(WaitHandle eventOccured, String eventName, Int32 timeout) 09:22:39 at CreatedTests.FileSystemWatcher_Created_WatcherDoesntFollowSymLinkToFile() 09:22:44 MoveFileTests.Unix_File_Move_In_Nested_Directory_Triggers_Event(eventType: Changed, moveRaisesEvent: False) [FAIL] 09:22:44 Should not observe a Changed event within 500ms 09:22:44 Expected: False 09:22:44 Actual: True 09:22:44 Stack Trace: 09:22:44 at Utility.ExpectNoEvent(WaitHandle eventOccured, String eventName, Int32 timeout) 09:22:44 at MoveFileTests.<>c__DisplayClass1.b__0(AutoResetEvent eventOccured, TemporaryTestDirectory ttd) 09:22:44 at Utility.TestNestedDirectoriesHelper(WatcherChangeTypes change, Action`2 action, NotifyFilters changeFilers) 09:22:44 at MoveFileTests.MoveAndCheck_NestedDirectory(WatcherChangeTypes eventType, Boolean moveRaisesEvent) 09:22:44 at MoveFileTests.Unix_File_Move_In_Nested_Directory_Triggers_Event(WatcherChangeTypes eventType, Boolean moveRaisesEvent) ``` cc: @sokket 3218 area-System.Net NameResolutionPalTests.GetHostByName_HostName test failing on OS X in CI The failure didn't repro for me locally, but it's happening consistently in the CI build: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_mac_debug_tst/25/console ``` System.Net.NameResolution.PalTests.NameResolutionPalTests.GetHostByName_HostName [FAIL] 22:46:36 System.ArgumentOutOfRangeException : Specified argument was out of the range of valid values. 22:46:36 Parameter name: newAddress 22:46:36 Stack Trace: 22:46:36 at System.Net.IPAddress..ctor(Int64 newAddress) 22:46:36 at System.Net.NameResolutionPal.CreateHostEntry(hostent* hostent) 22:46:36 at System.Net.NameResolutionPal.GetHostByName(String hostName) 22:46:36 at System.Net.NameResolution.PalTests.NameResolutionPalTests.GetHostByName_HostName() ``` 3219 area-System.Net Disable GetHostByName_HostName test failing on OSX #3218 3224 area-System.Linq Expression compiler - Y combinator issue "The following fails to compile correctly: Expression f = Expression.Lambda(Expression.Empty(), Expression.Parameter(typeof(X))); var a = Expression.Lambda(Expression.Invoke(f, f)); var m = typ.DefineMethod(""A"", MethodAttributes.Public | MethodAttributes.Static); a.CompileToMethod(m); where X is defined as: public delegate void X(X a); The object identity of 'f' used in the invocation expression seems the culprit due to some caching in the lambda compiler. " 3225 area-System.Security Investigate if Unix X509Chain.Build has the same truth table as Windows On Unix it seems to be returning false for any validation errors, and on Windows it might be more nuanced. 3226 area-System.Security Investigate if Unix and Windows X509Chains treat root certs in ExtraStore the same "Right now I think Unix says ""you told me about it, clearly it's trusted"" and Windows says ""it wasn't in one of the trust stores, it isn't trusted"". So the Unix version needs to be less trusting. But this behavior is currently unconfirmed. " 3232 area-System.Security Unix OpenSsl-based CryptographicExceptions should set both Message and HResult Currently only Message is set, making it hard to reason about the error (in cases where it makes sense to do so). This is the Unix half of #3062. 3233 area-System.Security Windows CNG/CAPI-based CryptographicExceptions should set both Message and HResult "In CoreFx the library that CryptographicException is in is platform agnostic, so it doesn't load a code for the int-only constructor. In debug builds the message should be of the form (`""(0x{0:X8}) {1}"", hr, Interop.mincore.GetMessage(hr))`; later that might change to all builds if we really like it. This is the Windows half of #3062. " 3239 area-System.IO Fixed OSX file creation time format FileInfo.CreationTime on OSX was returning a correctly timed but incorrectly formatted DateTime. This commit fixes this issue and adds coverage to a existing test to fill the test gap that let the issue through. 3243 area-System.Globalization Some tests failing due to changes in Globalization data We need to pay down some test debt here. There are a lot of places where we hard code expected values for tests which can change depending on the underlying OS which provides the data. We should figure out the right way to detect what OS we are on so we can provide the correct expected data. 3244 area-System.Linq Consider having expression interpreter available in all build flavors For some of our Bing service, we often run into the case where expression compilation is too expense, either because it's on a request/response path, or because it's statically known that the expression is small and will be evaluated only once. I can provide more details on such scenarios if needed. Unfortunately, expression interpretation is a property of the .NET build flavor, e.g. for .NET Native the Compile method uses the interpreter. It'd be nice to have an Interpret method alongside Compile to enable those scenarios. It'd also help a lot in writing parity tests for the expression compiler and interpreter because their outcomes can be cross-checked. 3245 area-System.Linq Add N-ary internal specializations for InvocationExpression "In Bing, we use an expression normal form whereby method call expressions etc. get erased for invocation expressions of an unbound parameter, e.g.: Expression.Call(where, xs, filter) becomes Expression.Invoke(Expression.Parameter(typeof(...), ""where""), xs, filter) This is done to decouple client libraries and service libraries through erasure of MemberInfo objects. The service uses a binder to bind the unbound parameter expression to a service-side definition and beta-reduces the invocation of the (lambda-based) definition. A lot of those invocation expressions have 0, 1, 2, 3 arguments. However, unlike MethodCallExpression, InvocationExpression has no N-ary specializations that help to reduce memory. The proposal is to add such specializations to conserve memory. " 3248 area-System.Net Setting HttpWebRequest.AutomaticDecompression When I look at the source of HttpWebRequest I see that the AutomaticDecompression property of the HttpClientHandler is forced to `DecompressionMethods.None` ([line 373](https://github.com/dotnet/corefx/blob/07f8c0fe64ac44bedced873181d6832bc2afcc50/src/System.Net.Requests/src/System/Net/HttpWebRequest.cs)). How do I set it properly? In the past we could simply do this: `request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;` 3249 area-System.Runtime Add a monotonic clock to the framework "**Problem:** Many BCL components use `DateTime.UtcNow` to compute timeouts and time deltas. When you run a ""Find Usages"" on `DateTime.UtcNow` in .NET Framework assemblies a lot of usage sites come to light. For example, in ADO.NET, transactions, caches, WebRequest, remoting, WCF, SignalR, security. **All of these usages are bugs** because the current system time can change significantly forwards and backwards at any time. This means that in case the system clock is changed many .NET components will spuriously fail. Common symptoms would be timeouts firing to early or too late (possibly never). Perf counters also might show too high or negative values. Sample scenario: 1. The system clock spuriously jumps forward due to misconfiguration by years. 2. A cache item is added. 3. The clock is fixed. Then, the cache item will essentially stay around forever if eviction is based on `DateTime.UtcNow`. I believe this bug exists in the BCL. User code also has the same problems. **Solution:** Add some kind of monotonic clock to the BCL. The main property of such a clock would be that its time advances linearly. It should never jump when the system clock changes. It should not exhibit split second jumps. It should behave reasonably in case the system sleeps or hibernates or in case the VM is paused. This monotonic clock class should be fast and convenient to use so that it is a no-brainer to switch all `DateTime.UtcNow` usages over to the new model. Maybe we can add `Environment.TickCount64` as well. `Environment.TickCount64` alone would not be good enough because it is awkward to use. I'm not sure if `Stopwatch` would cover all these requirements. It is a reference type which might be too heavy for hot code path in the BCL. Maybe we can add a new value type that is made exactly for this purpose. A sketch: ``` struct MonotonicTime { long Ticks; MonotonicTime operator + (MonotonicTime, TimeSpan); TimeSpan operator - (MonotonicTime, MonotonicTime); bool IsNegative; static MonotonicTime Current; ... } ``` To summarize, I request: 1. The framework timeouts should be fixed. (The TPL bugs apparently are already fixed. I reported these a while ago.) 2. Add `Environment.TickCount64`. 3. Provide a convenient, high-performance monotonic clock. " 3253 area-System.IO Support for tar archives Right now corefx supports zip files as well as gz files. Would it be hard to get it to support tar files as well for compatibility with the other OS's who package files as tgz very often? A C# implementation already exists at https://code.google.com/p/tar-cs/ and could be used either as a guideline or directly imported. If this is something that is desired I could work on designing an API, but it shouldn't be hard to visualize how it might look like. 3256 area-System.Net WinHttpWebSocketState is leaking a GCHandle System.Net.WebSockets.Client.WinHttpWebSocketState allocates a GCHandle in the constructor with this code: ``` c# private GCHandle _operationHandle = new GCHandle(); public WinHttpWebSocketState() { _operationHandle = GCHandle.Alloc(this); } ``` But if never calls Free() on the GCHandle. Normally, you need code like this to free the handle: ``` c# if (_operationHandle.IsAllocated) { _operationHandle.Free(); } ``` 3260 area-System.Console System.Console.Is[Input|Output]Redirected is not available in CoreFX IsInputRedirected and IsOutputRedirected properties are not available in System.Console 3262 area-System.Numerics Vector tests failing on one of my machines "On my local machine I doing a default build.cmd which builds x64 debug I keep getting the below failures. If it helps these pass on my other machine and the machine they fail on has the following processor: ""Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz, 3601 Mhz, 4 Core(s), 8 Logical Processor(s)"" cc @mellinoe @CarolEidt System.Numerics.Tests.Vector3Tests.Vector3CopyToTest [FAIL](TaskId:163) Assert.Throws() Failure (TaskId:163) Expected: typeof(System.ArgumentOutOfRangeException) (TaskId:163) Actual: typeof(System.NullReferenceException): Object reference not set to an instance of an object. (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\Vector3Tests.cs(28,0): at System.Numerics.Tests.Vector3Tests.<>c__DisplayClass1_0.b__1() (TaskId:163) System.Numerics.Tests.Vector4Tests.Vector4CopyToTest [FAIL](TaskId:163) Assert.Throws() Failure (TaskId:163) Expected: typeof(System.ArgumentOutOfRangeException) (TaskId:163) Actual: typeof(System.NullReferenceException): Object reference not set to an instance of an object. (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\Vector4Tests.cs(28,0): at System.Numerics.Tests.Vector4Tests.<>c__DisplayClass1_0.b__1() (TaskId:163) System.Numerics.Tests.Vector2Tests.Vector2CopyToTest [FAIL](TaskId:163) Assert.Throws() Failure (TaskId:163) Expected: typeof(System.ArgumentOutOfRangeException) (TaskId:163) Actual: typeof(System.NullReferenceException): Object reference not set to an instance of an object. (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\Vector2Tests.cs(28,0): at System.Numerics.Tests.Vector2Tests.<>c__DisplayClass1_0.b__1() (TaskId:163) System.Numerics.Tests.GenericVectorTests.GreaterThanOrEqualAnySByte [FAIL](TaskId:163) Assert.False() Failure (TaskId:163) Expected: False (TaskId:163) Actual: True (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(1248,0): at System.Numerics.Tests.GenericVectorTests.TestVectorGreaterThanOrEqualAny[T]() (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(1204,0): at System.Numerics.Tests.GenericVectorTests.GreaterThanOrEqualAnySByte() (TaskId:163) System.Numerics.Tests.GenericVectorTests.GreaterThanOrEqualAllSByte [FAIL](TaskId:163) Assert.True() Failure (TaskId:163) Expected: True (TaskId:163) Actual: False (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(1300,0): at System.Numerics.Tests.GenericVectorTests.TestVectorGreaterThanOrEqualAll[T]() (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(1259,0): at System.Numerics.Tests.GenericVectorTests.GreaterThanOrEqualAllSByte() (TaskId:163) System.Numerics.Tests.GenericVectorTests.CopyToByte [FAIL](TaskId:163) Assert.Throws() Failure (TaskId:163) Expected: typeof(System.ArgumentOutOfRangeException) (TaskId:163) Actual: typeof(System.NullReferenceException): Object reference not set to an instance of an object. (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(311,0): at System.Numerics.Tests.GenericVectorTests.<>c__DisplayClass100_0`1.b__1() (TaskId:163) System.Numerics.Tests.GenericVectorTests.CopyToSByte [FAIL] (TaskId:163) Assert.Throws() Failure (TaskId:163) Expected: typeof(System.ArgumentOutOfRangeException) (TaskId:163) Actual: typeof(System.NullReferenceException): Object reference not set to an instance of an object. (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(311,0): at System.Numerics.Tests.GenericVectorTests.<>c__DisplayClass100_0`1.b__1() (TaskId:163) System.Numerics.Tests.GenericVectorTests.CopyToUInt16 [FAIL](TaskId:163) Assert.Throws() Failure (TaskId:163) Expected: typeof(System.ArgumentOutOfRangeException) (TaskId:163) Actual: typeof(System.NullReferenceException): Object reference not set to an instance of an object. (TaskId:163) Stack Trace: (TaskId:163) D:\git\corefx\src\System.Numerics.Vectors\tests\GenericVectorTests.cs(311,0): at System.Numerics.Tests.GenericVectorTests.<>c__DisplayClass100_0`1.b__1() (TaskId:163) System.Numerics.Tests.GenericVectorTests.CopyToInt16 [FAIL](TaskId:163) Assert.Throws() Failure (TaskId:163) Expected: typeof(System.ArgumentOutOfRangeException) (TaskId:163) " 3266 area-System.Linq Use optimized N-ary MethodCallExpression nodes from all Call factories "Unlike Expression.Block, the overloads of Expression.Call that take in an Expression[] or IEnumerable do not attempt to check the number of supplied arguments in order to instantiate more compact expression representations than the standard [Instance]MethodCallExpressionN nodes. The language binds to the Expression[] overload. Various tools on our end, such as expression deserializers, also dispatch through the IEnumerable overload. It'd be good for those factories to analyze the number of supplied arguments and use compact representations if available. Also, [Instance]MethodCallExpression0 nodes are missing, even though it's quite common to have such expressions (e.g. ToString). Those could be added as part of this. Note that - in addition to this - it may be possible for the language compilers to check for the N-ary factory method overloads and bind to more specialized overloads when it finds them in the corefx libraries it's compiling against (much like other places where the compiler uses ""feature detection""). This would avoid the Expression[] allocation when it's not needed, therefore reducing some allocation costs. " 3268 area-System.Net CurlHandler : Ensure SendAsync cannot be called twice for the same request Originally discovered during the PR, ensure that CurlHandler's SendAsync cannot be called on the same HttpRequestMessage more than once. HttpClient's `SendAsync` already ensures that. We may need similar guarantee in `CurlHandler` as well. 3269 area-System.Net CurlHandler statusCode parsing - Error cases CurlHandler does StatusCode parsing in the http response message. Verify that CurlHandler behavior is identical to WinHttpHandler in case of negative scenarios. CurlHandler throws parse exception if status code in the Http response is not a valid status code. 3275 area-System.Collections Add a way to start enumeration of SortedSet and SortedDictionary in the middle I'd like an efficient way to start enumeration of a sorted collection (in particular `SortedSet` and `SortedDictionary`) at a specified value. This should work even if the value is not part of the collection. **Why is it useful?** Such a feature would be useful for many things. For example: - Finding all elements in an interval. - Finding the closest element to an item in the set - With a properly chosen comparer prefixes sort immediately before their extensions. So this could be used to find the longest prefix that's in the collection. - Could be used to map from a continuous variable (e.g. time) to a value. For example if a system can either be `on` or `off` at a particular time, one could put the switching evens in a `SortedDictionary` and then query it to figure out if the system was `on` at a particular time. - You could save the current element, mutate the collection (invalidating existing enumerators) and then create a new iterator starting at that element. **Different approaches** One way to implement this feature is having a pair of functions that return an enumerable starting at a particular point in the collection. I'm not sure what the name should be, possible choices include `UpwardsFrom`/`DownwardsFrom`, `From`/`ReverseFrom`, `ForwardsFrom`/`BackwardsFrom`. An alternative approach would be extending `SortedSet.Enumerator` adding a public constructor that takes a start element and a flag for backward iteration, but I don't like that overly low level approach. **A concrete API suggestion** on `SortedSet` ``` public IEnumerable UpwardsFrom(T start); public IEnumerable DownwardsFrom(T start); ``` on `SortedDictionary` ``` public IEnumerable> UpwardsFrom(TKey start); public IEnumerable> DownwardsFrom(TKey start); ``` and optionally on `SortedDictionary.KeyCollection` ``` public IEnumerable UpwardsFrom(TKey start); public IEnumerable DownwardsFrom(TKey start); ``` **Test cases** ``` var sortedSet = new SortedSet{1, 3, 5}; sortedSet.UpwardsFrom(2) // [3, 5] sortedSet.UpwardsFrom(3) // [3, 5] sortedSet.UpwardsFrom(6) // [] sortedSet.DownwardsFrom(4) // [3, 1] sortedSet.DownwardsFrom(3) // [3, 1] sortedSet.DownwardsFrom(0) // [] ``` **Demo implementation** Their behaviour would match the following extension methods, but would avoid the linear scan for the first match: ``` public static IEnumerable UpwardsFrom(this SortedSet sortedSet, T start) { var comparer = sortedSet.Comparer; return sortedSet.SkipWhile(item => comparer.Compare(item, start) < 0); } public static IEnumerable DownwardsFrom(this SortedSet sortedSet, T start) { var comparer = sortedSet.Comparer; return sortedSet.Reverse().SkipWhile(item => comparer.Compare(item, start) > 0); } ``` 3278 area-System.IO Fixed test bug with FileInfo_GetSetTimes resolves #3272 3281 area-System.Diagnostics ProcessTests.TestWorkingSet64 failed on OS X in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_mac_debug_tst/31/console ``` 21:51:43 System.Diagnostics.ProcessTests.ProcessTests.TestWorkingSet64 [FAIL] 21:51:43 Assert+WrapperXunitException : File path: d:\j\workspace\dotnet_corefx_windows_debug\src\System.Diagnostics.Process\tests\ProcessTests.cs. Line: 339 21:51:43 ---- Assert.True() Failure 21:51:43 Expected: True 21:51:43 Actual: False 21:51:43 Stack Trace: 21:51:43 at Assert.WrapException(Exception e, String callerFilePath, Int32 callerLineNumber) 21:51:43 at Assert.True(Boolean condition, String userMessage, String path, Int32 line) 21:51:43 at System.Diagnostics.ProcessTests.ProcessTests.TestWorkingSet64() 21:51:43 ----- Inner Stack Trace ----- 21:51:43 at Assert.True(Boolean condition, String userMessage, String path, Int32 line) ``` 3282 area-System.Security CtorTests.InvalidCertificateBlob failed on OS X in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_mac_debug_tst/31/console ``` 21:55:11 System.Security.Cryptography.X509Certificates.Tests.CtorTests.InvalidCertificateBlob [FAIL] 21:55:11 Assert.Equal() Failure 21:55:11 Expected: 218595386 21:55:11 Actual: -2146233088 21:55:11 Stack Trace: 21:55:11 at System.Security.Cryptography.X509Certificates.Tests.CtorTests.InvalidCertificateBlob() ``` 3290 area-System.Xml XmlWriterTestModule.TCCloseOutput failing in outer loop http://dotnet-ci.cloudapp.net/job/dotnet_corefx_outerloop_windows_release/703/console http://dotnet-ci.cloudapp.net/job/dotnet_corefx_outerloop_windows_debug/705/console ``` TestCase:XmlWriterSettings: CloseOutput - File Not Found: writer.out Check that underlying stream is NOT CLOSED when CloseOutput = FALSE and Create(TextWriter) unexpected exception happend:Object reference not set to an instance of an object. at XmlCoreTest.Common.FilePathUtil.getStream(String filename) in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\Common\tests\SystemXml\XmlCoreTest\FilePathUtil.cs:line 173 at XmlWriterAPI.Test.TCCloseOutput.CloseOutput_1() in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\System.Xml.ReaderWriter\tests\Writers\XmlWriterApi\XmlFactoryWriterTests.cs:line 2258 at OLEDB.Test.ModuleCore.CVariation.Execute() in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\Common\tests\SystemXml\ModuleCore\cvariation.cs:line 74 at OLEDB.Test.ModuleCore.CTestCase.Execute() in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\Common\tests\SystemXml\ModuleCore\ctestcase.cs:line 69 FAILED File Not Found: writer.out Check that underlying stream is NOT CLOSED when CloseOutput = FALSE and Create(Stream) unexpected exception happend:Object reference not set to an instance of an object. at XmlCoreTest.Common.FilePathUtil.getStream(String filename) in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\Common\tests\SystemXml\XmlCoreTest\FilePathUtil.cs:line 173 at XmlWriterAPI.Test.TCCloseOutput.CloseOutput_1() in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\System.Xml.ReaderWriter\tests\Writers\XmlWriterApi\XmlFactoryWriterTests.cs:line 2258 at OLEDB.Test.ModuleCore.CVariation.Execute() in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\Common\tests\SystemXml\ModuleCore\cvariation.cs:line 74 at OLEDB.Test.ModuleCore.CTestCase.Execute() in d:\j\workspace\dotnet_corefx_outerloop_windows_debug\src\Common\tests\SystemXml\ModuleCore\ctestcase.cs:line 69 FAILED ``` 3292 area-System.Net POST'ing empty content throws an error in System.Net.Http If I make a web request and get an HTTP 204 in return, `WinHttpRequestStream` throws an `IOException` with a success message as the inner exception. It looks like a bug in `WinHttpRequestStream.WriteData`. `Interop.WinHttp.WinHttpWriteData` returns false if the response is a 204 (I guess because there is no content). `Marshal.GetLastError` returns `0` though indicating that there was no error. Should it check if `GetLastError() == 0` rather than checking the result of `Interop.WinHttp.WinHttpWriteData`? I don't know if this is an issue with WinHttp or System.Net.Http, but I don't get this behavior on the version of the dll in Nuget. 3293 area-System.Collections System.Collections.Immutable assembly considered Security Critical I'm writing a program where some untrusted code is running in a separate app domain. I want this untrusted code to use various Immutables, but I get the following error: `Attempt by security transparent method 'MyMethod' to access security critical method 'System.Collections.Immutable.ImmutableList.CreateBuilder()' failed.` `Assembly 'MyAssembly, Version=1.0.0.0, Culture=neutral, PublicKeyToken=**** is partially trusted, which causes the CLR to make it entirely security transparent regardless of any transparency annotations in the assembly itself. In order to access security critical code, this assembly must be fully trusted.` Why is the Immutable assembly considered security critical? I would imaging this is an oversight and it should be Security Transparent. 3294 area-System.Console Enabling System.Console.WindowWidth and System.Console.CursorVisible properties in CoreFx We are building a cross-platform command line tool which has support for displaying tables (using ascii chars) and showing progress. To render the table correctly we use WindowWidth and want to hide the cursor when we show progress. 3297 area-System.Linq Improve optimization strategy for BlockExpression The check for optimization to N-ary specialized nodes in the Block factories only happens for the Expression[] overload. With the addition of IReadOnlyList<T> to the framework, we can also optimize the IEnumerable<Expression> overloads, similar to the changes proposed in #3288. 3301 area-Serialization Reconcile XmlSerializerImplementation in XmlSerializer with desktop In NetCore: public abstract class XmlSerializerImplementation { public virtual XmlSerializationReader Reader { get { throw new NotSupportedException(); } } public virtual XmlSerializationWriter Writer { get { throw new NotSupportedException(); } } public virtual IDictionary ReadMethods { get { throw new NotSupportedException(); } } public virtual IDictionary WriteMethods { get { throw new NotSupportedException(); } } public virtual IDictionary TypedSerializers { get { throw new NotSupportedException(); } } public virtual bool CanSerialize(Type type) { throw new NotSupportedException(); } public virtual XmlSerializer GetSerializer(Type type) { throw new NotSupportedException(); } } Desktop: public abstract class XmlSerializerImplementation { public virtual XmlSerializationReader Reader { get { throw new NotSupportedException(); } } public virtual XmlSerializationWriter Writer { get { throw new NotSupportedException(); } } public virtual Hashtable ReadMethods { get { throw new NotSupportedException(); } } public virtual Hashtable WriteMethods { get { throw new NotSupportedException(); } } public virtual Hashtable TypedSerializers { get { throw new NotSupportedException(); } } public virtual bool CanSerialize(Type type) { throw new NotSupportedException(); } public virtual XmlSerializer GetSerializer(Type type) { throw new NotSupportedException(); } } 3302 area-Serialization Perf tests that deserialize large object graphs 3307 area-System.Net Allow UseDefaultCredentials property for HttpClientHandler on unix 3311 area-Infrastructure Need a standard convention to refer to bugs from corresponding regression tests. Roslyn uses WorkItemAttribute as a standardized way for regression tests to refer to bugs. It has been found very useful in the past, since bug descriptions and discussions can contain a lot of relevant information, that would be hard to make a part of a test. Being an attribute makes it easy to find in an automated way if so needed (bugs migrated to alternative store or something) Turns out Linq.Expression does not have any convention for regression tests. I suggested commented out attribute as a temporary measure to not lose the information about corresponding bug, but there seem to be a need for a better, more standard convention for this. 3312 area-System.Net WinHttpHandlerTest.SendAsync_SlowPostRequestWithTimedCancellation_ExpectTaskCanceledException failed in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_release/1240/console http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_release/1245/console ``` System.Net.Http.WinHttpHandlerUnitTests.WinHttpHandlerTest.SendAsync_SlowPostRequestWithTimedCancellation_ExpectTaskCanceledException [FAIL] Assert.Equal() Failure Expected: 0 Actual: 3 Stack Trace: d:\j\workspace\dotnet_corefx_windows_release\src\System.Net.Http.WinHttpHandler\tests\UnitTests\WinHttpHandlerTest.cs(37,0): at System.Net.Http.WinHttpHandlerUnitTests.WinHttpHandlerTest.Dispose() at ReflectionAbstractionExtensions.DisposeTestClass(ITest test, Object testClass, IMessageBus messageBus, ExecutionTimer timer, CancellationTokenSource cancellationTokenSource) ``` 3315 area-Serialization Fix performance issue when deserializing base64 content with linebreaks Porting Desktop fix to NetCore for performance issue when deserializing base64 content with linebreak characters. This changes improves the throughput to 50x for 1KB input and more than 130x for 10KB to 1MB inputs. Fix #3316 3316 area-Serialization Fix performance issue when deserializing base64 content with linebreaks chars in DCS 3317 area-System.Reflection Merge master to dev/metadata Conflicts: src/System.Reflection.Metadata/src/System/Reflection/Metadata/MetadataReader.cs src/System.Reflection.Metadata/src/project.lock.json 3318 area-System.Runtime Proposal: Base64/128 Variable Length Quantity (VLQ) in System.Convert "### Concept: `System.Convert` currently resides in CoreCLR repo: [https://github.com/dotnet/coreclr/../System/Convert.cs](https://github.com/dotnet/coreclr/blob/f665f226/src/mscorlib/src/System/Convert.cs). The class has number of methods `ToBase64*` and `FromBase64*` with various signatures. The base64 encoding has 8-bit fixed length. The are many applications of variable length base64 encoding; Variable Length Quantity (VLQ). IMO, one of the most popular application of B64 VLQ is in [V3 source maps](https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k/edit?pli=1#!); which is used by browsers, editors, trans-compilers etc. to obtain source-to-source mappings in trans-compilation scenarios. For instance; CoffeeScript, Less, Sass, Stylus, SweetJS and TypeScript are languages, that compile into CSS or JavaScript. Today, the developer tools in all major browsers are capable of mapping generated source to original by the virtue of source-maps. Additionally, Base128 VLQ has applications in media formats, such as MIDI, XMF etc. [[1](https://en.wikipedia.org/wiki/Variable-length_quantity)] ### Proposed API: ``` c# namespace System { public static class Convert { [...] public static byte[] FromBaseNVLQString( EncodingRadix radix; string s, BaseNVLQSetting setting); public static byte[] FromBaseNVLQCharArray( EncodingRadix radix; char[] inArray, Int32 offset, Int32 length, BaseNVLQSetting setting); public static string ToBaseNVLQString( EncodingRadix radix; byte[] inArray, BaseNVLQSetting setting); public static string ToBaseNVLQString( EncodingRadix radix; byte[] inArray, int offset, int length, BaseNVLQSetting setting); public static unsafe string ToBaseNVLQString( EncodingRadix radix; byte[] inArray, int offset, int length, BaseNVLQSetting setting); public static int ToBaseNVLQCharArray( EncodingRadix radix; byte[] inArray, int offsetIn, int length, char[] outArray, int offsetOut, BaseNVLQSetting setting); public static unsafe int ToBaseNVLQCharArray( EncodingRadix radix; byte[] inArray, int offsetIn, int length, char[] outArray, int offsetOut, BaseNVLQSetting setting); public static unsafe int ToBaseNVLQCharArray( EncodingRadix radix; byte[] inArray, int offsetIn, int length, char[] outArray, int offsetOut, BaseNVLQSetting setting); } public enum EncodingRadix { Base64 = 64; Baase128 = 128; } public struct BaseNVLQSetting { private int _base, _baseShift, _baseMask. _continuationBit; private char[] separators; public BaseNVLQSetting( int base, int baseShift, int baseMask, int continuationBit, char[] separators); public enum Template { V3SourceMap, MIDI, XMF } public static BaseNVLQSetting FromTemplate(BaseNVLQSetting.Template template) { switch(template) { case Template.V3SourceMap: return new BaseNVLQSetting(1 << 5, 5, 4, 1 << 5, new char[] {',', ';'}); break; [...] default: throw new Exception(""Unknown template.""); } } } } ``` ### Working Example: We implemented sourcemap encoding and decoding in WE2013: [https://github.com/madskristensen/WebEssentials2013/../Base64VLQ.cs](https://github.com/madskristensen/WebEssentials2013/blob/96d37799/EditorExtensions/Shared/Helpers/Base64VLQ.cs), inspired by Mozilla's JavaScript implementation: https://github.com/mozilla/source-map. " 3319 area-System.Net CurlHandler asserting in WCF tests ``` ---- DEBUG ASSERTION FAILED ---- ---- Assert Short Message ---- If disposed, the stream must also be completed. ---- Assert Long Message ---- at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) at System.Diagnostics.Debug.Assert(Boolean condition, String message) at System.Net.Http.CurlHandler.CurlResponseStream.VerifyInvariants() at System.Net.Http.CurlHandler.CurlResponseStream.SignalComplete(Exception error) at System.Net.Http.CurlHandler.EasyRequest.Cleanup() at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(EasyRequest completedOperation, Int32 messageResult) at System.Net.Http.CurlHandler.MultiAgent.WorkerLoop() at System.Net.Http.CurlHandler.MultiAgent.<>c.b__13_0(Object s) at System.Threading.Tasks.Task.Execute() at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot) at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) ``` 3320 area-System.Net "CurlHandler throwing ""HTTP stack not implemented"" exception from SupportsRedirectConfiguration" https://github.com/dotnet/corefx/blob/master/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs#L25 3321 area-System.Net "CurlHandler throwing ""HTTP stack not implemented"" from MaxRequestContentBufferSize" https://github.com/dotnet/corefx/blob/master/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs#L129-L133 3322 area-System.Net CurlHandler throwing NotImplementedException for non-http/https schemes https://github.com/dotnet/corefx/blob/master/src/System.Net.Http/src/System/Net/Http/Unix/CurlHandler.cs#L276-L279 Even if we're supposed to throw here, it shouldn't be NotImplemented.ByDesignWithMessage, more likely PlatformNotSupportedException. That said, this might be dead code: it looks like checks for the sheme happen before we even get to this point: ``` Unhandled Exception: System.ArgumentException: Only 'http' and 'https' schemes are allowed. Parameter name: requestUri at System.Net.Http.HttpRequestMessage.InitializeValues(HttpMethod method, Uri requestUri) at System.Net.Http.HttpRequestMessage..ctor(HttpMethod method, Uri requestUri) at System.Net.Http.HttpClient.GetAsync(Uri requestUri, HttpCompletionOption completionOption, CancellationToken cancellationToken) at System.Net.Http.HttpClient.GetContentAsync[T](Uri requestUri, HttpCompletionOption completionOption, T defaultValue, Func`2 readAs) at System.Net.Http.HttpClient.GetStringAsync(Uri requestUri) at System.Net.Http.HttpClient.GetStringAsync(String requestUri) ``` So potentially this might just need to be changed to an assert. 3323 area-System.Net Disable SafeHandle leak checking in WinHttpHandler Unit Tests Addresses issue #3312 I've discovered that the FakeSafeWinHttpHandle leak detection isn't stable. SafeHandles finalize via the CriticalFinalizer thread. This thread doesn't seem to run deterministically, so using code like this to wait for SafeHandles to be finalized (and then testing for leaks) doesn't work reliably. ``` c# public static void ForceGarbageCollection() { // Make several passes through the FReachable list. for (int i = 0; i < 10; i++) { GC.Collect(); GC.WaitForPendingFinalizers(); } } ``` For now, logging will be used to verify no leaks. 3324 area-Meta Hardware Details API - Hardware Fingerprint It would be great to add an api that we could use to get hardware specific information. Some of these may include 1. Mac Addresses 2. Processor Information 3. Graphic card Information 4. Base Board Information 5. Unique fingerprint based on hardware I think many applications out there implement features depending on what hardware is available. Furthermore like us, licensing applications in certain scenarios is a must, and I am sure that many applications out there currently, make use of hardware information to create a unique fingerprint which is calculated from hardware information. 3326 area-System.Net Remove delegate allocation from WinHttpHandler.SendAsync Each SendAsync call was allocating a new delegate for StartRequest. cc: @davidsh, @CIPop 3327 area-System.Net Implement HttpClientHandler.MaxRequestContentBufferSize on Unix Changed to match behavior on Windows (get returns 0, set throws PlatformNotSupportedException). Fixes #3321 cc: @kapilash, @davidsh, @CIPop 3328 area-System.Net Implement CurlHandler.SupportsRedirectConfiguration It does. Return true. Fixes #3320 cc: @kapilash 3329 area-System.Net Client Certificate Options for Client Handler When HttpClientHandler.ClientCertificateOption is set to ClientCertificateOptions.Automatic, curl handler passes on the X509 certificates from X509 store to libcurl Fixes #3151. 3333 area-System.Net Remove faulty assert from CurlHandler.CurlResponseStream CurlResponseStream.VerifyInvariants has an assert that assumes that if the stream has been disposed, it must have also completed. But since this stream is exposed publically, it's possible for consuming code to Dispose of the stream before the request has finished, violating the assert. In such a situation, a subsequent attempt to transfer data to the response stream will fail and cause the request to be aborted as desired, cleaning up resources (there are probably some opportunities here for some optimizations to handle this case, but that can be explored later.) This commit just removes the bad assert. Fixes #3319 cc: @kapilash, @roncain 3334 area-System.Security Linux & Mac Support for System.Security.Cryptography.Pkcs Is there any plan to support classes in System.Security.Cryptography.Pkcs namespace? Its essential to create p7s files and sign data. Thanks 3340 area-System.Net Port additional System.Net.Http tests to CoreFx Added more http tests to open source from the original internal ToF tests. Some of the tests should be refactored in the future to avoid Assert'ing multiple logical things. Fixed up test namespaces for System.Net.Http and System.Net.Http.WinHttpHandler. 3343 area-System.Net HttpContentTest.ReadAsStringAsync_SetInvalidCharset_ThrowsInvalidOperationException fails on Linux Part of PR #3340, this test fails on Linux. Disabling for now. System.Net.Http.Functional.Tests.HttpContentTest.ReadAsStringAsync_SetInvalidCharset_ThrowsInvalidOperationException Stacktrace MESSAGE: Assert.Throws() Failure Expected: typeof(System.InvalidOperationException) Actual: (No exception was thrown) +++++++++++++++++++ STACK TRACE: --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() at System.Net.Http.Functional.Tests.HttpContentTest.d__bd.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 3345 area-System.Net Net Secure Xplat Implementation 1. Abstracted the code as per design shared earlier. 2. Made SecureChannel & SSPIWrapper agnostic of os. 3. Created 1 class 1 file for each new classes & move them to windows specific folder. 4. Added new interfaces & modules related to Cert. 5. SSPINegoInterface & SSPIAuthType, currently are not part of csproj as suggested by cipop. Have included unix Interop & related code as well for completeness. You can choose to review Windows first as well. 3349 area-Infrastructure Declare requirements in the tests to run\skip based on runtime checks As of now a no. of tests have requirements which make them applicable only in certain situations. For example, ServiceProcess tests can only be run in admin mode, several PLINQ tests need to be run on multi-threaded machines. (We do have an option of marking these tests OpenLoop, but that seems like a broader classification). Currently, the only way to assert these requirements is to update the constructor to check for the requirements, causing each individual test to fail. It would be good to have a means to skip\run tests based on a runtime check. #3134 has more details 3356 area-System.Net Restrict the SSL handshake test to 3 seconds. When things don't go so well this test takes significantly longer than an acceptable waiting period. On success it's fractions of a second. 3360 area-System.Net Correct broken System.Net.Http solution file. When the folders were adjusted for test prep, the folder searched for the functional tests was adjusted, but the file name needed to be updated to match too. 3362 area-System.Net System.Net.Security related Pending PR coments/TODO 1. Interop.OpenSsl TODO - See if SSL_CTX_Set_quite_shutdown can be used 3364 area-System.Threading ReaderWriterLockSlim.AcquireWriterLock timing out does not wake up waiting readers The [documentation for `ReaderWriterLockSlim.TryEnterWriteLock`](https://msdn.microsoft.com/en-us/library/bb339812.aspx) states: > While threads are blocked waiting to enter write mode, additional threads that try to enter read mode or upgradeable mode **block until all the threads waiting to enter write mode have either timed out or** entered write mode and then exited from it. (emphasis mine). However, the implementation does not behave like the bolded part of the documentation states. Imagine the following scenario with three threads (_T1_, _T2_, _T3_): 1. _T1_ acquires a read lock. 2. _T2_ tries to acquire a write lock and blocks, waiting for a timeout (as _T1_ holds the lock). 3. _T3_ tries to acquire a read lock and blocks (because _T2_ is blocked waiting for the write lock, and per the documentation, this means all further readers are blocked until _T2_ times out). 4. _T2_’s timeout expires. Per the documentation, _T3_ should now wake up and acquire the read lock. However, this does not happen and _T3_ is blocked forever (or, until some other event, e.g. _T1_ releasing the read lock, causes the implementation to wake up waiting threads). [Reproducible code for this scenario is available on StackOverflow](http://stackoverflow.com/a/32656459/304138), with some discussion. AFAICT, this is a simple bug caused by the fact that `TryEnterWriteLock` blocks further attempts to acquire a read lock, but when leaving by timeout, it does not wake them up. Specifically, I’d guess the `ExitMyLock` call in [`ReaderWriterLockSlim’s WaitOnEvent`](https://github.com/dotnet/corefx/blob/master/src/System.Threading/src/System/Threading/ReaderWriterLockSlim.cs#L923) should have been `ExitAndWakeUpAppropriateWaiters`. The same bug seems to exist for the non-slim `ReaderWriterLock`. 3365 area-System.Net Dummy PR with xplat SslStream fixes Sharing some fixes found during unit testing. We will be sending these changes as part of official PRs. The idea is to unblock devs who may be hitting issues while testing. Note that the first 2 commits are from https://github.com/dotnet/corefx/pull/3345 3369 area-System.Net Incorporated some comments - Net Secure Xplat Implementation Incorporated some of the comments. Sending it as a new PR as original PR will be closed, once we merge into dev_sslstream to unblock partners. Tracking all issues via #3362 . (I am yet to updated the issue with complete list). Updated all TODO's in the code to refer this issue. This also includes fixes from Vijay's dummy PR https://github.com/dotnet/corefx/pull/3365. 3374 area-System.Security Add OuterLoop tests for X509Store We currently don't have tests for adding and removing certificates to an X509Store on the file system on Unix. We should add some. 3375 area-System.Net Add System.Net.Security to README.md cc: @stephentoub @CIPop @SidharthNabar 3382 area-System.Security GetECDSAPrivateKey() methods not being tested. This requires building a PFX which then triggers other testing problems (race conditions due to the temporary key being stored on disk.) 3383 area-System.Security X509Certificate2.GetECDsaPublicKey() on Unix still unimplemented. (from Jeremy) As for GetECDsaPublicKey, I think that you can get there with EVP_PKEY\* X509_PUBKEY_get(X509_PUBKEY_). Probably want to just add it to the native interop library as EVP_PKEY_ GetX509EvpPublicKey(X509\* x509). Then you can just unpack the EC_KEY the same way you did for the private key here. 3384 area-System.Globalization Investigate casing performance We should write some performance tests for casing and then play around with breaking up the loops in the implementation of the native ChangeCase method use on cross platform to see if we can make things faster. See the conversation [on the PR](https://github.com/dotnet/coreclr/pull/1604#discussion-diff-40141703) for more details. 3386 area-System.Net Various fixes for Unix SslStream @vijaykota had put out a PR into the dev-sslstream branch which gets an SslStream acting as a client working enough to connect to https://www.microsoft.com/, send it a malformed request, and read the reply. This PR is just a integrated version of #3365's non-experimental commits. 3389 area-System.Globalization Decouple CultureData from tests. Test data differs across OSes and so we need a way to select the expected data for globalization tests depending what windows version we are on or what version of the CLDR we are using. 3390 area-System.Net Add support for ECDsa certificates on Unix SslStream The initial version is being added with RSA only, because you have to start somewhere. 3391 area-System.Globalization TestDateTimeParsingWithSpecialCultures failing on ICU for sr-Latn-XX As part of b1995378c63df05117c57505c50f46e256c32b3a Tarek added some additional cultures to TestDateTimeParsingWithSpecialCultures in System.Runtime. These additional tests don't pass when run with ICU. ``` MESSAGE: Assert.True() Failure Expected: True Actual: False +++++++++++++++++++ STACK TRACE: at DateTimeTests.TestDateTimeParsingWithSpecialCultures(String cultureName) ``` 3394 area-System.Net HttpClient error on dnx-coreclr-linux-x64.1.0.0-beta8-15653 System.TypeInitializationException: The type initializer for 'System.Net.Http.CurlHandler' threw an exception. ---> System.TypeInitializationException: The type initializer for 'libcurl' threw an exception. ---> System.TypeInitializationException: The type initializer for 'HttpInitializer' threw an exception. ---> System.DllNotFoundException: Unable to load DLL 'System.Net.Http.Native': The specified module could not be found. (Exception from HRESULT: 0x8007007E) at Interop.HttpInitializer.EnsureCurlIsInitialized() at Interop.HttpInitializer..cctor() --- End of inner exception stack trace --- at Interop.HttpInitializer.Initialize() at Interop.libcurl..cctor() --- End of inner exception stack trace --- at Interop.libcurl.curl_version_info(Int32 curlVersionStamp) at System.Net.Http.CurlHandler..cctor() --- End of inner exception stack trace --- at System.Net.Http.CurlHandler..ctor() at System.Net.Http.HttpClientHandler..ctor() at ConsoleApp1.Program.d__7.MoveNext() 3406 area-System.Globalization IDNA tests conform to windows behavior, not specs The windows implementation of IDNA2008 does not follow the relevant specs in some cases (by design, for compatibility purposes). Our tests assume this behavior and then fail when run with ICU's implementation of ToAscii and ToUnicode. We need to decouple the tests into a core of tests which have the same behavior on both Windows and XPlat and then an additional set of tests which have different results and are platform specific. 3418 area-System.Runtime (Discussion) Case insensitive strings as a type I have read #649, and it seems to be a special case of a generic problem. Consider system where some strings are case sensitive (e.g. hash), and some aren't (e.g. name). Currently .NET defaults to being case sensitive, so I have to remember which string is which in all use cases. And those cases are not only about `s1.Equals(s2)` -- it is also every call to `ToDictionary`, every creation of `HashSet` and various other cases and optimizations that are really hard to keep track of. It would be great if I could define string as being case-insensitive on type level, e.g. use `CaseInsensitiveString`. This would affect Comparer selection wherever this type is used as a key, and all comparisons. Not quite sure whether it makes general sense and what the API would be like, so just putting it here as a discussion point. 3419 area-System.Runtime StringBuilder.AppendJoin (appending lists to StringBuilder) "#### Background Let's say you want to build a string that contains a separated list (e.g. list of filenames, messages, or anything else). If it is the only thing you want, you can use `string.Join`. However what if you want to append something else? If you `Concat` with the result of `string.Join`, that means that you unnecessarily allocated an intermediate joined string, which can be rather large (depending on the original collection). On the other hand, `StringBuilder` would require something like: ``` csharp var builder = new StringBuilder(""Names: ""); for (var i = 0; i < names.Length; i++) { if (i > 0) { // this becomes even more involved if `names` is `IEnumerable` builder.Append("", ""); } builder.Append(names[i]); } ``` That can be abstracted into an extension method, however I believe that extension would be general and useful enough to benefit from being in NetFX. #### API Design (updated based on API review, second pass -- see https://github.com/dotnet/corefx/issues/3419#issuecomment-273254581) ``` csharp public class StringBuilder { public StringBuilder AppendJoin(string separator, params object[] values); public StringBuilder AppendJoin(string separator, IEnumerable values); public StringBuilder AppendJoin(string separator, params string[] value); public StringBuilder AppendJoin(chat separator, params object[] values); public StringBuilder AppendJoin(char separator, IEnumerable values); public StringBuilder AppendJoin(char separator, params string[] value); // string.Join overloads NOT taken: //public StringBuilder AppendJoin(string separator, IEnumerable values); //public StringBuilder AppendJoin(string separator, string[] values, int startIndex, int count); //public StringBuilder AppendJoin(char separator, IEnumerable values); //public StringBuilder AppendJoin(char separator, string[] values, int startIndex, int count); // REJECTED in the first API review round: //public StringBuilder AppendJoin(string separator, T[] values, int startIndex, int count); //public StringBuilder AppendJoin(string separator, params T[] values); //public StringBuilder AppendJoin(char separator, params T[] values); } ``` Note: Motivation for the API approval update is described in https://github.com/dotnet/corefx/issues/3419#issuecomment-273254581 -- adjusting to follow proven working form of 'string.Join', correctly working with most-used overload calls - see usage example in https://github.com/dotnet/corefx/issues/5552#issuecomment-268395367. #### Compatibility Potential source-level issue if someone is already using an extension method with the same name. " 3421 area-System.Globalization DateTimeFormatInfoGetInstance.PosTest2 failed on Linux in CI "http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst_prtest/2662/console ``` 03:20:10 System.Globalization.Tests.DateTimeFormatInfoGetInstance.PosTest2 [FAIL] 03:20:10 Assert.Equal() Failure 03:20:10 Expected: DateTimeFormatInfo { AMDesignator = ""AM"", AbbreviatedDayNames = [""Sun"", ""Mon"", ""Tue"", ""Wed"", ""Thu"", ...], AbbreviatedMonthGenitiveNames = [""Jan"", ""Feb"", ""Mar"", ""Apr"", ""May"", ...], AbbreviatedMonthNames = [""Jan"", ""Feb"", ""Mar"", ""Apr"", ""May"", ...], Calendar = GregorianCalendar { CalendarType = Localized, Eras = [...], IsReadOnly = True, MaxSupportedDateTime = 9999-12-31T23:59:59.9999999, MinSupportedDateTime = 0001-01-01T00:00:00.0000000, ... }, ... } 03:20:10 Actual: DateTimeFormatInfo { AMDesignator = ""de."", AbbreviatedDayNames = [""V"", ""H"", ""K"", ""Sze"", ""Cs"", ...], AbbreviatedMonthGenitiveNames = [""jan."", ""febr."", ""m��rc."", ""��pr."", ""m��j."", ...], AbbreviatedMonthNames = [""jan."", ""febr."", ""m��rc."", ""��pr."", ""m��j."", ...], Calendar = GregorianCalendar { CalendarType = Localized, Eras = [...], IsReadOnly = False, MaxSupportedDateTime = 9999-12-31T23:59:59.9999999, MinSupportedDateTime = 0001-01-01T00:00:00.0000000, ... }, ... } 03:20:10 Stack Trace: 03:20:11 at System.Globalization.Tests.DateTimeFormatInfoGetInstance.PosTest2() 03:20:11 Finished: System.Globalization.Tests ``` " 3423 area-System.Data API for retrieving Columns Schema from DbDataReader This is to unwrap the specific issue of resultset schemas from the many involved in #1039. [Back on May 5th](https://github.com/dotnet/corefx/issues/1039#issuecomment-99186515) @davkean noted: > We want GetSchemaTable in some form (we should get a separate bug filed on that, however, as you could imagine bringing that back with a different design that doesn't pull in DataSet/DataTable). While it doesn't seem that anyone cares _what method_ exists for of getting the schema of a resultset, we seem to agree on needing _a method_ of doing it in .Net Core. For example, if a user runs a simple query such as: ``` SQL SELECT 'bob' Name, 1 Id ``` Then there's no way to see what the types are. While getting the .Net type ([via `.GetFieldType()`](https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqldatareader.getfieldtype%28v=vs.110%29.aspx), e.g. `DateTime`) is possible and getting _part_ the SQL type ([via `.GetDataTypeName()`](https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqldatareader.getdatatypename%28v=vs.110%29.aspx), e.g. `NVARCHAR`) is possible, other vital attributes of the schema shape currently aren't possible. For example there's no way to distinguish a `varchar(20)` from a `varchar(max)`. There's also no way to tell if the column was nullable. Previously, this functionality was provided by [`.GetSchemaTable()`](https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqldatareader.getschematable%28v=vs.110%29.aspx) which returned a [`DataTable`](https://msdn.microsoft.com/en-us/library/system.data.datatable%28v=vs.110%29.aspx) class. Currently I have a very simple use case: query data here, create a table with that data elsewhere. The query can be something like: `SELECT * FROM sys.database_files`. I point this out to illustrate that `INFORMATION_SCHEMA` (as proposed by some in #1039 as an alternative) is not a viable option. The same issue presents on any stored procedure which can also return any result set shape depending on the code paths. We need a way to see the scheme _of a result set_, these database-query alternatives are both expensive and non-covering. This is a hard blocker for projects of myself and others. I have uses in both one-off projects as well as [Opserver](https://github.com/opserver/Opserver) (which monitors SQL) and [Dapper.Net](https://github.com/StackExchange/dapper-dot-net). I know from #1039 that I'm not alone on this block. The portability of .Net core is _extremely_ appealing for certain projects, especially those where we need to monitor the innards of SQL without depending on a .Net install (many if not most existing SQL servers only have .Net 3.5). Unfortunately though, this one missing piece of the puzzle may prevent using it altogether, forcing full CLR installs. While SqlClient specifically serves my blocking use case, it's important to point out as @FransBouma [notes in the other issue](https://github.com/dotnet/corefx/issues/1039#issuecomment-142202400): this needs to be in the base API, not specifically up in SqlClient. For example, I also need this for Dapper features and it works across many providers - it is not at all SQL Server specific. So how about it, can we please get an API for result set schemas added back? I'm happy to contribute to designing and provide feedback on a new API here if that's the best route. I know many others are as well. 3425 area-System.Net Testing SslStream Read/Write logic 3431 area-System.Net Unix SslStream needs to verify the remote cert is valid for the destination address 3432 area-System.Net Adjust the PAL separation for certificate verification The Windows version requires that there be a valid handle to the native chain representation; the Unix version never has a valid native handle. So move the PAL separation to a slightly higher abstraction level, and let each side party on it in their own way. 3439 area-Infrastructure Replace Microsoft.DotNet.CoreCLR dependency with Microsoft.NETCore.Runtime.CoreCLR Remove dotnet-coreclr feed from dir.props and NuGet.config once this is done as well. See https://github.com/dotnet/corefx/pull/3413#discussion_r40368451 cc @weshaggard 3441 area-System.Net Follow up on CR feedback for System.Net.Sockets. See #3422. 3444 area-System.Net [SslStream] CheckX509Hostname wildcard matching may be too restrictive "[RFC 2595](http://www.ietf.org/rfc/rfc2595.txt) says: ``` - A ""*"" wildcard character MAY be used as the left-most name component in the certificate. For example, *.example.com would match a.example.com, foo.example.com, etc. but would not match example.com. ``` [RFC 2818](http://www.ietf.org/rfc/rfc2818.txt) says: ``` Matching is performed using the matching rules specified by [RFC2459]. If more than one identity of a given type is present in the certificate (e.g., more than one dNSName name, a match in any one of the set is considered acceptable.) Names may contain the wildcard character * which is considered to match any single domain name component or component fragment. E.g., *.a.com matches foo.a.com but not bar.foo.a.com. f*.com matches foo.com but not bar.com. ``` Where did the `f*.com` => `foo.com` example come from? " 3446 area-System.Net [Feature] CheckX509Hostname does not match IDNA Internationalized domains are very likely going to confuse this function. 3447 area-System.Net UDPClientTest.BeginSend_AsyncOperationCompletes_Success failed in CI on Windows http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/4229/console ``` Discovering: System.Net.Sockets.APMServer.Tests Discovered: System.Net.Sockets.APMServer.Tests Starting: System.Net.Sockets.APMServer.Tests System.Net.Sockets.Tests.UDPClientTest.BeginSend_AsyncOperationCompletes_Success [FAIL] System.Net.Sockets.SocketException : An address incompatible with the requested protocol was used Stack Trace: d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets\src\System\Net\Sockets\Socket.cs(3211,0): at System.Net.Sockets.Socket.DoBeginSendTo(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, EndPoint endPointSnapshot, SocketAddress socketAddress, OverlappedAsyncResult asyncResult) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets\src\System\Net\Sockets\Socket.cs(3156,0): at System.Net.Sockets.Socket.BeginSendTo(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, EndPoint remoteEP, AsyncCallback callback, Object state) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets\src\System\Net\Sockets\UDPClient.cs(598,0): at System.Net.Sockets.UdpClient.BeginSend(Byte[] datagram, Int32 bytes, IPEndPoint endPoint, AsyncCallback requestCallback, Object state) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets.Legacy\tests\FunctionalTests\UdpClientTest.cs(51,0): at System.Net.Sockets.Tests.UDPClientTest.BeginSend_AsyncOperationCompletes_Success() Finished: System.Net.Sockets.APMServer.Tests ``` 3450 area-System.Net 516 system.net Removing custom error checking from System.Net.* (ignoring System.Net.Sockets for the moment) @stephentoub - I'm planning on doing a squash/overall comment like for #3020 . 3463 area-System.Net Enable SslStream for Unix "Merging the current status of dev-sslstream into master. - The Windows version should continue to work just fine. - The Unix version has received sanity testing, and has limited functionality, but is believed to be in a ""basic scenarios, for integration purposes"" state. - `stream.AuthenticateAsClient(string)` :white_check_mark: - `stream.AuthenticateAsClientAsync(string)` :white_check_mark: - `stream.AuthenticateAsServer(X509Certificate)` :ok: (requires the provided instance be X509Certificate2, and using RSA) - `stream.Write(byte[])` :white_check_mark: - `stream.Read(byte[], int, int)` :white_check_mark: - {Read|Write}Async :grey_question: (it's expected to work, but wasn't explicitly verified) - Off the golden path may be a bit rough; unexpected disconnects and the like may have different exception flows than their Windows counterparts (if encountered, please report!) - Mutual authentication / client authentication certificates :x: " 3464 area-System.Net HttpClient isn't always canceling in response to cancellation requests "We first found this issue with a WCF negative test that expected a TimeOutException when setting a ‘SendTimeout’ on our BasicHttpBinding and calling a service operation that took longer to return. Previously this test passed with the elapsed time being just a little longer than the ‘SendTimeout’ value. As of a couple weeks ago we still got the TimeOutException but the elapsed time seemed to correlate to however long it took for the call to return from the service operation. So if the ‘SendTimeout’ was set to 5 milliseconds and the service operation waited 30 milliseconds to return, the elapsed time would be something over 30 milliseconds. As best we could determine it has something to do with the CancellationTokenSource perhaps when it is creating the linked token source and passing it to the SetTimeout method. (HttpClient.cs SendAsync method) We isolated this issue using no WCF code with the following client side code… ``` csharp Stopwatch watch = new Stopwatch(); HttpClient client = new HttpClient(); HttpRequestMessage message = new HttpRequestMessage(); message.RequestUri = new Uri(""http://localhost:54464/Home/About""); var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); try { watch.Start(); var responseTask = client.SendAsync(message, cts.Token); responseTask.Wait(); } catch (Exception ex) { Debug.WriteLine(ex.Message + ""\n"" + ex.StackTrace); } finally{ watch.Stop(); } Assert.InRange(watch.ElapsedMilliseconds, 8000, 15000); ``` In this example we are just calling the ‘About’ page of an ASP.NET MVC application (default VS template) and just added the following in the HomeController.About() method… ``` csharp public ActionResult About() { ViewBag.Message = ""Your application description page.""; Thread.Sleep(30*1000); return View(); } ``` **NOTE: I have no idea why but it seems to work as expected when in debug mode, the issue reproduces when executed normally.** " 3467 area-System.Globalization Investigate ICU EndsWith when string has zero collation elements at the end In StartsWith we ignore zero collation elements at the start of the string, we probably need to do the same thing in EndsWith. 3468 area-System.Globalization Implement IndexOfOrdinal and LastIndexOfOrdinal in the ICU Shim The naive implementation of these methods in CompareInfo.Unix.cs is correct but allocates both to do a substring and to case the string when ignore case is true. We should just just implement these methods in the Shim itself. It can walk the string without allocating and do casing inline without having to round trip to managed code for each character. 3474 area-System.Security Add opt-in setting for ensuring strong random numbers There have been some reports that Linux and OpenSSL in particular may not use enough entropy in all scenarios https://www.blackhat.com/docs/us-15/materials/us-15-Potter-Understanding-And-Managing-Entropy-Usage-wp.pdf We need to ensure we are properly seeding OpenSSL's PRNG. There are some conflicting articles/sites that say using /dev/urandom is sufficient: http://www.2uo.de/myths-about-urandom/ http://security.stackexchange.com/questions/7718/openssl-rand-poll-good-enough http://security.stackexchange.com/questions/3259/howto-seed-the-prng-in-openssl-properly?rq=1 One idea is to use the new [getrandom()](http://man7.org/linux/man-pages/man2/getrandom.2.html) syscall, which will block if there isn't enough entropy in the system. When that isn't available, we need to decide whether we should: 1. block 2. fail 3. continue 3475 area-System.Security System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildChain failure on OSX in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_mac_debug_tst/42/testReport/System.Security.Cryptography.X509Certificates.Tests/ChainTests/BuildChain/ 3476 area-System.Net Two OSX HttpClientHandlerTest Failures in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_mac_debug_tst/42/testReport/System.Net.Http.Functional.Tests/HttpClientHandlerTest/GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock/ http://dotnet-ci.cloudapp.net/job/dotnet_corefx_mac_debug_tst/42/testReport/System.Net.Http.Functional.Tests/HttpClientHandlerTest/GetAsync_CredentialIsNetworkCredentialUriRedirect_StatusCodeUnauthorized/ 3477 area-System.Net Fixes #3464 where http requests don't honor cancellation Add a registration for the passed in cancellation token which cancels the CancellationTokenSource.Token that is returned to the caller. 3480 area-System.Data Make interfaces as the official ADO.NET Provider API instead of classes From what I can see currently on the [corefx-progress page for System.Data.Common](https://github.com/dotnet/corefx-progress/blob/master/src-diff/System.Data.Common.md), the interfaces (IDbCommand, IDbConnection, etc) were removed in favor of the usage of abstract classes. But in the new API, most of the main methods are not virtual or abstract. On DbCommand alone we can see this: ``` cs public DbConnection Connection { get; set; } public DbParameterCollection Parameters { get; } public DbTransaction Transaction { get; set; } public DbParameter CreateParameter(); public Task ExecuteNonQueryAsync(); public DbDataReader ExecuteReader(); public DbDataReader ExecuteReader(CommandBehavior behavior); public Task ExecuteReaderAsync(); public Task ExecuteReaderAsync(CommandBehavior behavior); public Task ExecuteReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken); public Task ExecuteReaderAsync(CancellationToken cancellationToken); public Task ExecuteScalarAsync(); ``` While these methods can certainly be made virtual or abstract, it would be much more useful to have the real interfaces back, and make any public API depend on these interfaces instead of the abstract classes. This is mostly useful when developing libraries. Today it's very hard to mock a datareader to make it return a specific value for testing purposes. The same for ensuring that ExecuteReaderAsync was called, not ExecuteReader, etc. I propose the provider factory instead should be made as an interface: ``` cs public interface IDbProviderFactory { IDbCommand CreateCommand(); IDbConnection CreateConnection(); IDbConnectionStringBuilder CreateConnectionStringBuilder(); IDbParameter CreateParameter(); } ``` And then follow from there to the rest of the provider to things like `IDbDataReader`, `IDbTransaction`, etc. We know the interfaces became out of sync for some reason in the past and the abstract classes were made the official API, but this doesn't need to be the case anymore in corefx. Note that this doesn't mean removing the System.Data.Common in any way, but instead make the Common classes implement these interfaces, and you wouldn't use System.Data.Common unless you're implementing the provider. Applications would depend only on the interfaces instead. Please consider this to make the API more testable on corefx 1.0. Related to discussions on #1039 and #3423. 3481 area-System.Linq How best to implement Linq's iterators. The current approach with `System.Linq.Enumerator`'s enumerables and their enumerators copies the approach for compiler-generated iterators; the same instance that serves as the enumerable serves as the result for the first call to `GetEnumerator()`, so avoiding an extra allocation. There is a race inherent to this approach; if two threads called `GetEnumerator()` at the same time they could end up with the same object when they should have separate instances. This is resolved by capturing `Environment.CurrentManagedThreadId` on construction and only using `this` as the result of `GetEnumerator()` if it matches a second call. This relatively expensive call is of no value if: 1. `GetEnumerator()` is never called (many optimisations skip it in various ways). 2. `GetEnumerator()` is repeatedly called (after the first call). 3. `GetEnumerator()` is only called once, on another thread (e.g. if a query is constructed before an `await` and enumerated after it). #3313 changes the approach to changing the state with `Interlocked.CompareExchange()` to resolve the race in a different way. This means no call in case 1 and 3 above, but is more expensive in case 2. It's about even in the case of one `GetEnumerator()` at least as some tests with Win64 DNX suggest (`CompareExchange` being about twice the case of `CurrentManagedThreadId` and being called half as often). In the discussion on that pull request, @Vsadov points out that the optimisation of avoid an allocation may not be as valuable as it once was in any case. Some very limited experimentation suggests that indeed just creating a new object on every call to `GetEnumerator()` is a clear win. There are a few possible variants here. We can have the same classes that server as both enumerable and enumerator, or separate classes (some reduction in state held, some increase in jitted code). If separate classes are used, internal calls to `GetEnumerator()` can be moved to this object's construction, further reducing state held in some cases, but being an observable change if that call throws an exception. A completely different possibility is to use the current approach, but avoid calling CurrentManagedThreadId is some cases. Some experimentation is in order. Some possible approaches: 1. Leave things as they are. 2. Use the basic approach currently used, but avoiding the `CurrentManagedThreadId` call, https://github.com/JonHanna/corefx/tree/fewer_threadid_calls 3. Use `Interlocked.CompareExchange()`: https://github.com/JonHanna/corefx/tree/defer_obtaining_threadid 4. Use the same classes as currently, but always allocating for `GetEnumerator()`: https://github.com/JonHanna/corefx/tree/fresh_enumerators 5. Use separate enumerators https://github.com/JonHanna/corefx/tree/separate_enumerators_cautious 6. ~~Use separate enumerators, shifting `GetEnumerator()` calls (observable change) https://github.com/JonHanna/corefx/tree/separate_enumerators~~ I'm planning to do some comparisons with these variants, but I'm opening this issue ahead of that for suggestions as to situations the comparisons must cover. In particular: Is 6 (with the observable change) completely out of the question? What tests would one expect avoiding an allocation to do better than not avoiding it? (First brief experiments suggest it's always a lose, but I'm wary of removing what was clearly intended as an optimisation if I can't see where it succeeded in optimising, though maybe it's just a matter of older CLR versions). 3482 area-System.Collections Proposal: (I)Dictionary.GetValueOrDefault ## Scenario In almost every app or library I write I almost always end up needing to write code like this, in at least one place: ``` c# SomeClass returnValue; return DictionaryOfSomeClass.TryGetValue(someKey, out returnValue) ? returnValue : null; ``` The reason that the `IDictionary` contract throws from its indexer for a non-existent key are well-known: an alternative behavior of returning `null` is nonsensical if `TValue` is a value type. Returning `default(TValue)` would work, but would clearly be a misleading behavior. But with dictionaries of class-types it is often desirable and expected to return `null` for a non-existent key. ## Proposal ```c# namespace System.Collections.Generic { public class Dictionary { public TValue GetValueOrDefault(TKey key); public TValue GetValueOrDefault(TKey key, TValue defaultValue); } public static class CollectionExtensions { public static TValue GetValueOrDefault(this IDictionary dictionary, TKey key); public static TValue GetValueOrDefault(this IDictionary dictionary, TKey key, TValue defaultValue); public static TValue GetValueOrDefault(this IReadOnlyDictionary dictionary, TKey key); public static TValue GetValueOrDefault(this IReadOnlyDictionary dictionary, TKey key, TValue defaultValue); } } ``` **Notes:** - There will be source-compile problem with ambiguity in all projects that already have such custom extension method `GetValueOrDefault` on `IDictionary` or `IReadOnlyDictionary`. Which is general problem for any extension method we add into CoreFX. ### Original Proposal - replaced by the one above I propose the following extension method (I don't like the name `GetValueNoThrow`, so that can be ignored for now). Please note that it applies only when `TValue` is a class, so it does not violate any behavior for value types. ``` c# public static class ClassyDictionaryExtensions { public static TValue GetValueNoThrow_Ignore_The_Name(this IDictionary dictionary, TKey key) where TValue : class { if (dictionary == null) { throw new ArgumentNullException(nameof(dictionary)); } TValue value; dictionary.TryGetValue(key, out value); return value; } } ``` The new code would end up with one very-easy-to-read line of code: ``` c# return DictionaryOfSomeClass.GetValueNoThrow(someKey); ``` ## Discussion Would others find such an extension method useful? 3483 area-System.Console Add an API to execute a console program and capture its output in a simple way Clearly, we can use the `Process` class to run a program, redirect its output and capture the output. But doing so correctly is unfathomably complicated: 1. You need to read both StdOut and StdErr. Failing to do so can result in hangs. 2. Output events can arrive _after_ WaitForExit has returned. (Update: This concern does not exist.) 3. Output events signal their completing by sending one `null` string at the end. 4. `myProcess.StandardOutput.ReadAsync` throws an exception for the last read with a confusing message (http://stackoverflow.com/questions/32800802/unable-to-read-output-from-child-process-from-c-sharp-app-using-async-methods#32800802). 5. Reading binary output requires contortions (`myProcess.StandardOutput.BaseStream`) where it is not clear whether this is supported at all. We are reaching into the `BaseStream` property which might expose an internal object not meant for consumption. 6. Failing to quickly read the redirected streams stalls the child process. 7. Many more on this list: http://csharptest.net/321/how-to-use-systemdiagnosticsprocess-correctly/ 8. Probably, even more that are on nobodies list. This API is terrible! Please provide an easy and reliable way to capture output. Here's a strawman for a simple API `IEnumerable Process::StartWithOutputRedirected()` Now, this clearly is insufficient and probably not nice API design but you get the point. The following goals should be achieved: 1. Little potential for mistakes. 2. Streaming output for unbounded output size. 3. Text and binary output. 4. Differentiate between StdOut and StdErr. For example by returning an `IEnumerable` of a DTO like `struct ConsoleOutput { string Data; ConsoleOutputType Type; /*Output, Error*/ }`. 5. The output stream should end at exactly the same point in time that WaitForExit ends at. Delay both to only complete when the other also has completed. 6. The API should allow reading in a non-linewise fashion. 7. A simplified version that just captures all output on both channels into one `string`. This is probably what most consumers want anyway. 3484 area-System.Net Some asserts during X-Plat Sockets Tests See: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst/1021/console. The Debug.Assert here did not cause the test to fail. ``` ---- DEBUG ASSERTION FAILED ---- ---- Assert Short Message ---- SafeCloseSocket::ReleaseHandle(handle:0) ---- Assert Long Message ---- ReleaseHandle failed. at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) at System.Diagnostics.Debug.Fail(String message, String detailMessage) at System.Net.GlobalLog.Assert(String message, String detailMessage) at System.Net.GlobalLog.Assert(Boolean condition, String messageFormat, Object[] data) at System.Net.Sockets.SafeCloseSocket.InnerSafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.DangerousRelease() at System.Net.Sockets.SafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.InternalFinalize() at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing) at System.Runtime.InteropServices.SafeHandle.Finalize() at System.Net.DebugSafeHandleMinusOneIsInvalid.Finalize() ---- DEBUG ASSERTION FAILED ---- ---- Assert Short Message ---- SafeCloseSocket::ReleaseHandle(handle:0) ---- Assert Long Message ---- ReleaseHandle failed. at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) at System.Diagnostics.Debug.Fail(String message, String detailMessage) at System.Net.GlobalLog.Assert(String message, String detailMessage) at System.Net.GlobalLog.Assert(Boolean condition, String messageFormat, Object[] data) at System.Net.Sockets.SafeCloseSocket.InnerSafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.DangerousRelease() at System.Net.Sockets.SafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.InternalFinalize() at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing) at System.Runtime.InteropServices.SafeHandle.Finalize() at System.Net.DebugSafeHandleMinusOneIsInvalid.Finalize() ---- DEBUG ASSERTION FAILED ---- ---- Assert Short Message ---- SafeCloseSocket::ReleaseHandle(handle:0) ---- Assert Long Message ---- ReleaseHandle failed. at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) at System.Diagnostics.Debug.Fail(String message, String detailMessage) at System.Net.GlobalLog.Assert(String message, String detailMessage) at System.Net.GlobalLog.Assert(Boolean condition, String messageFormat, Object[] data) at System.Net.Sockets.SafeCloseSocket.InnerSafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.DangerousRelease() at System.Net.Sockets.SafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.InternalFinalize() at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing) at System.Runtime.InteropServices.SafeHandle.Finalize() at System.Net.DebugSafeHandleMinusOneIsInvalid.Finalize() Finished: System.Net.Sockets.APMServer.Tests === TEST EXECUTION SUMMARY === System.Net.Sockets.APMServer.Tests Total: 231, Errors: 0, Failed: 0, Skipped: 0, Time: 13.856s ---- DEBUG ASSERTION FAILED ---- ---- Assert Short Message ---- SafeCloseSocket::ReleaseHandle(handle:0) ---- Assert Long Message ---- ReleaseHandle failed. at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) at System.Diagnostics.Debug.Fail(String message, String detailMessage) at System.Net.GlobalLog.Assert(String message, String detailMessage) at System.Net.GlobalLog.Assert(Boolean condition, String messageFormat, Object[] data) at System.Net.Sockets.SafeCloseSocket.InnerSafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.DangerousRelease() at System.Net.Sockets.SafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.InternalFinalize() at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing) at System.Runtime.InteropServices.SafeHandle.Finalize() at System.Net.DebugSafeHandleMinusOneIsInvalid.Finalize() Finished: System.Net.Sockets.AsyncServer.Tests === TEST EXECUTION SUMMARY === System.Net.Sockets.AsyncServer.Tests Total: 231, Errors: 0, Failed: 0, Skipped: 0, Time: 13.955s ---- DEBUG ASSERTION FAILED ---- ---- Assert Short Message ---- SafeCloseSocket::ReleaseHandle(handle:0) ---- Assert Long Message ---- ReleaseHandle failed. at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) at System.Diagnostics.Debug.Fail(String message, String detailMessage) at System.Net.GlobalLog.Assert(String message, String detailMessage) at System.Net.GlobalLog.Assert(Boolean condition, String messageFormat, Object[] data) at System.Net.Sockets.SafeCloseSocket.InnerSafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.DangerousRelease() at System.Net.Sockets.SafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.InternalFinalize() at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing) at System.Runtime.InteropServices.SafeHandle.Finalize() at System.Net.DebugSafeHandleMinusOneIsInvalid.Finalize() ---- DEBUG ASSERTION FAILED ---- ---- Assert Short Message ---- SafeCloseSocket::ReleaseHandle(handle:0) ---- Assert Long Message ---- ReleaseHandle failed. at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage) at System.Diagnostics.Debug.Fail(String message, String detailMessage) at System.Net.GlobalLog.Assert(String message, String detailMessage) at System.Net.GlobalLog.Assert(Boolean condition, String messageFormat, Object[] data) at System.Net.Sockets.SafeCloseSocket.InnerSafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.DangerousRelease() at System.Net.Sockets.SafeCloseSocket.ReleaseHandle() at System.Runtime.InteropServices.SafeHandle.InternalFinalize() at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing) at System.Runtime.InteropServices.SafeHandle.Finalize() at System.Net.DebugSafeHandleMinusOneIsInvalid.Finalize() ``` 3490 area-System.Net Incorporated some comments - Net Secure Xplat Implementation This is same as https://github.com/dotnet/corefx/pull/3369 Comments have already been addressed/replied. Any new comments can be given here. Since it's pending for roughly ~6 days, would like a quick closure. Thanks. Creating a PR to dotnet/master as we will no longer be using dev-sslstream. I will squash the commits once,I get the sign off. 3494 area-System.Net ConnectV4MappedIPAddressToV4Host_Success failed in CI on Linux http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst_prtest/2785/console ``` 09:31:53 System.Net.Sockets.Tests.DualMode.ConnectV4MappedIPAddressToV4Host_Success [FAIL] 09:31:53 System.Net.Sockets.SocketException : Unknown error 10054 09:31:53 Stack Trace: 09:31:54 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) 09:31:54 at System.Net.Sockets.Socket.Connect(EndPoint remoteEP) 09:31:54 at System.Net.Sockets.Socket.Connect(IPAddress address, Int32 port) 09:31:54 at System.Net.Sockets.Tests.DualMode.DualModeConnect_IPAddressToHost_Helper(IPAddress connectTo, IPAddress listenOn, Boolean dualModeServer, Int32 port) 09:31:54 at System.Net.Sockets.Tests.DualMode.ConnectV4MappedIPAddressToV4Host_Success() ``` 3495 area-System.Net SocketPerformance_SingleSocketClientAPM_LocalHostServerAPM failed in CI on Windows http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug/1351/console ``` 11:22:58 System.Net.Sockets.Performance.Tests.SocketPerformanceAPMTests.SocketPerformance_SingleSocketClientAPM_LocalHostServerAPM [FAIL] 11:22:58 Test execution is expected to be shorter than 5000 but was 7543 11:22:58 Expected: True 11:22:58 Actual: False 11:22:58 Stack Trace: 11:22:58 d:\j\workspace\dotnet_corefx_windows_debug\src\Common\tests\System.Net\Sockets\Performance\SocketPerformanceTests.cs(53,0): at System.Net.Sockets.Performance.Tests.SocketPerformanceTests.ClientServerTest(Int32 port, SocketImplementationType serverType, SocketImplementationType clientType, Int32 iterations, Int32 bufferSize, Int32 socket_instances, Int64 expectedMilliseconds) 11:22:58 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Sockets.Legacy\tests\PerformanceTests\SocketPerformanceAPMTests.cs(38,0): at System.Net.Sockets.Performance.Tests.SocketPerformanceAPMTests.SocketPerformance_SingleSocketClientAPM_LocalHostServerAPM() ``` cc: @davidsh, @cipop, @pgavlin 3497 area-System.Net SendPacketsElement_FilePart_Success failed in CI on Windows http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_release_prtest/4322/console ``` Starting: System.Net.Sockets.AsyncServer.Tests System.Net.Sockets.Tests.SendPacketsAsync.SendPacketsElement_FilePart_Success [FAIL] Timed out Expected: True Actual: False Stack Trace: d:\j\workspace\dotnet_corefx_windows_release_prtest\src\System.Net.Sockets.Legacy\tests\FunctionalTests\SendPacketsAsync.cs(397,0): at System.Net.Sockets.Tests.SendPacketsAsync.SendPackets(SendPacketsElement[] elements, SocketError expectedResut, Int32 bytesExpected) d:\j\workspace\dotnet_corefx_windows_release_prtest\src\System.Net.Sockets.Legacy\tests\FunctionalTests\SendPacketsAsync.cs(317,0): at System.Net.Sockets.Tests.SendPacketsAsync.SendPacketsElement_FilePart_Success() ``` cc: @davidsh, @cipop, @pgavlin 3498 area-System.Net System.Net changes for README.md - Update System.Net.Security description - Add System.Net.Sockets description 3499 area-System.Reflection Merge master to dev/metadata Straight PR from master -> dev/metadata (no conflicts) 3501 area-Infrastructure Add static and runtime verification tools against existing tests /cc: @stephentoub @vijaykota @pgavlin @davidsh We should augment our CI with static and runtime verification tools that can be enabled against our existing tests: Windows: - [ ] FXCop (edit: covered by https://github.com/dotnet/corefx/issues/14095) - [ ] PREfast for Native code - [ ] [AppVerifier](http://www.microsoft.com/en-us/download/details.aspx?id=20028) Linux: - [ ] [Valgrind](http://valgrind.org/) This should already be part of normal pre-commit procedures for all teams. Could we make this an automated part of CI/Weekly testing? 3502 area-System.Security Windows: new X509Certificate2(byte[]).GetRSAPrivateKey returns RSACSP instead of RSACng "The idea/thought was that we'd use CNG for everything except for hardware providers that were linked through CAPI. In particular, this would prevent the SHA256 lookup failure that occurs when using CAPI's PROV_RSA_FULL instead of PROV_RSA_AES. I'm pretty sure that we made the software keys be returned via RSACng on desktop (4.6). This was accomplished by calling CryptAcquireCertificatePrivateKey with the ""ncrypt only"" flag (see the code on [reference source](http://referencesource.microsoft.com/#System.Core/System/Security/Cryptography/X509Certificates/X509Native.cs,2e55137aa586849f)), since that can up-convert software based CAPI keys. " 3503 area-System.Reflection Merge master to dev/metadata Replacing #3499. Still no conflicts, but this time using an intermediate branch so that changes to master do not automatically become part of the PR and reset CI. 3504 area-System.Net ReceiveFromAsyncV4BoundToAnyV4_Success failure in CI on Windows http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug/1361/console ``` System.Net.Sockets.Tests.DualMode.ReceiveFromAsyncV4BoundToAnyV4_Success [FAIL] 18:41:42 System.TimeoutException : The operation has timed out. 18:41:42 Stack Trace: 18:41:42 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Sockets.Legacy\tests\FunctionalTests\DualModeSocketTest.cs(1836,0): at System.Net.Sockets.Tests.DualMode.ReceiveFromAsync_Helper(IPAddress listenOn, IPAddress connectTo, Int32 port) 18:41:42 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Net.Sockets.Legacy\tests\FunctionalTests\DualModeSocketTest.cs(1766,0): at System.Net.Sockets.Tests.DualMode.ReceiveFromAsyncV4BoundToAnyV4_Success() ``` cc: @davidsh, @CIPop, @pgavlin 3505 area-System.Diagnostics Process.GetProcessesByName significantly slower on Linux than Windows for nonexistent process "The performance test being run: ``` [Benchmark] [InlineData(1)] [InlineData(2)] [InlineData(3)] public void GetProcessesByName(int innerIterations) { foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) { for (int i = 0; i < innerIterations; i++) { Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); Process.GetProcessesByName(""1""); } } } ``` Linux perf results (38.455 total seconds): ``` ``` Windows perf results (3.355 total seconds): ``` ``` Further increasing the number of calls to GetProcessesByName makes comparatively little difference on Windows. For example, with an `InnerIterations` of 5000 (45000 total function calls), the elapsed time on Windows is only 106 seconds. In that same time Linux can only complete ~20 InnerIterations (180 total function calls). This suggests the Linux implementation takes roughly 250 times as long as the Windows implementation. " 3506 area-Infrastructure Can't run [Outerloop] tests using msbuild of individual test project "I frequently go to a single test directory to run tests just for that project. I.e., this will run just innerloop tests: > cd src\System.Net.Http\tests\FunctionalTests > msbuild /t:clean;build;test Now, say, I want to run `[Outerloop]` tests: I tried using this from the directory of the individual csproj file: `msbuild /t:clean;build;test /p:RunTestsWithCategories=Outerloop` or this: `msbuild /t:clean;build;test ""/p:XunitOptions=-trait category=outerloop""` and neither works. I have a vague memory of this working before. So, has this changed lately? " 3508 area-System.Net Change S.N.Sockets/tests to use (mostly) anonymous ports. This change only touches tests in System.Net.Sockets/tests; those under S.N.Sockets.Legacy/tests will be taken care of in a follow-up commit. 3510 area-System.Net Disabling System.Net.Sockets tests intermittently failing in CI. Disabling intermittently failing Sockets tests in CI. Related #3497 #3447 3512 area-System.IO Path.GetDirectoryName slower on Linux than Windows Core "The performance test being run: ``` [Benchmark] [InlineData(10000)] [InlineData(20000)] [InlineData(30000)] public void GetDirectoryName(int innerIterations) { string testPath = PerfUtils.GetTestFilePath(); foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i < innerIterations; i++) { Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); Path.GetDirectoryName(testPath); } } ``` Linux perf results (99.804 seconds): ``` ``` Windows Core perf results (4.147 seconds): ``` ``` " 3516 area-System.IO Modified PathInternal.Unix for great perf increase resolves #3515, #3515 3517 area-System.Reflection Add TypeNameParser tests @davkean Adding tests for your type name parser. I marked you as author on the commit with your original VS unit tests, then followed it up with commits to port to xunit. The port is minimal, and I have not for example bothered to convert to [Theory] etc. yet. I've also separated out the part that uses desktop-only API to compare against reflection type name parsing in to the .Desktop test project. Mind you, that project is temporary until we get test explorer working with portable tests, so I'm not sure we can keep that long term. cc @tmat Fix #534 3518 area-System.Collections Dictionary int constructor ~10 times faster on Windows than on Linux "Perf test being run: ``` [Benchmark] [InlineData(0)] [InlineData(1024)] [InlineData(4096)] [InlineData(16384)] public void ctor_int(int size) { foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i <= 5000; i++) { new Dictionary(size); new Dictionary(size); new Dictionary(size); new Dictionary(size); new Dictionary(size); new Dictionary(size); new Dictionary(size); new Dictionary(size); new Dictionary(size); } } ``` Linux results: ``` ``` Windows results: ``` ``` Note that the because of the way the test is written, destructing of the dictionaries can be included in the measured portion. It would be worthwhile to store local copies of the dictionaries so that destruction doesn't happen within the iteration.StartMeasurement `using` so the exact problem can be narrowed down. " 3519 area-System.Net Use anonymous ports in S.N.Sockets.Legacy/test/FunctionalTests. This is a follow-up to #3508. With this change, all of the legacy functional tests use anonymous ports. This also includes a small change to `SocketPerformanceTests.cs` to address PR feedback from #3508. 3520 area-System.Threading System.Threading.Tasks.Dataflow.Tests.EtwTests.TestEtw failed in CI on OSX This is a deterministic failure as far as I can tell. ``` MESSAGE: Assert.Equal() Failure Expected: 0 Actual: 2 +++++++++++++++++++ STACK TRACE: at System.Threading.Tasks.Dataflow.Tests.EtwTests.<>c__DisplayClass7.b__1() at System.Diagnostics.Tracing.TestEventListener.RunWithCallback(Action`1 handler, Action body) at System.Threading.Tasks.Dataflow.Tests.EtwTests.TestEtw() ``` http://dotnet-ci.cloudapp.net/job/dotnet_corefx_mac_debug_tst/50/testReport/junit/System.Threading.Tasks.Dataflow.Tests/EtwTests/TestEtw/ 3524 area-System.Data Incorporate Misc code review comments for SqlClient Please refer PR https://github.com/dotnet/corefx/pull/2986 The items to be taken care of, 1. Use utility methods. 2. Remove unnecessary base class calls 3. There is Obsolete SMI code which can be removed. Followup on the SMI v1 support and clean this up. 3528 area-System.Net Using SafeHandles instead of IntPtrs in Interop.libssl Changed the Unix code to use SafeHandles cc: @stephentoub @bartonjs @rajansingh10 @shrutigarg 3532 area-System.Net SendPacketsAsync.BufferZeroCountThenNormal_ZeroCountIgnored failed in CI on Windows http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/4396/console ``` System.Net.Sockets.Tests.SendPacketsAsync.BufferZeroCountThenNormal_ZeroCountIgnored [FAIL] Timed out Expected: True Actual: False Stack Trace: d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets.Legacy\tests\FunctionalTests\SendPacketsAsync.cs(231,0): at ``` cc: @davidsh, @cipop, @pgavlin 3533 area-System.Threading Timer_Change_BeforeDueTime_ChangesWhenTimerWillFire failed in CI http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug/1368/console ``` TimerChangeTests.Timer_Change_BeforeDueTime_ChangesWhenTimerWillFire [FAIL] 05:19:31 Should have received a timer event after this new duration 05:19:31 Expected: True 05:19:31 Actual: False 05:19:31 Stack Trace: 05:19:31 d:\j\workspace\dotnet_corefx_windows_debug\src\System.Threading.Timer\tests\TimerChangeTests.cs(94,0): at TimerChangeTests.Timer_Change_BeforeDueTime_ChangesWhenTimerWillFire() ``` 3534 area-Infrastructure $(CommonPath) in paths causing bad Visual Studio experience. "A bunch of our .csproj files have lines like <Compile Include ""$(CommonPath)\blah\blah""> where $(CommonPath) is defined in dir.prop. This has a really bad effect on VS experience as its navigation and Intellisense engine struggle to find these files. Defining $(CommonPath) inside the individual .csproj fixes the Intellisense and still gives us some measure of encapsulation, albeit imperfect. Is the encapsulation of ""Common"" really worth this? " 3535 area-System.Data Refactor and consolidate the AdapterUtil.cs Reuse the common functions from AdapterUtil.cs The AdapterUtil is present in both Data.Common and Data.SqlClient with common functions. Refactor the AdapterUtil.cs so that most of it can reside in Data.Common 3537 area-System.Text Investigate System.Text.Encodings.Web.Tests encoding failure using XUnit Theory "We currently have this TODO in our UnicodeHelpersTests: ``` C# // This loop should instead be implemented as a [Theory] with multiple [InlineData]s. // However, until globalization support is implemented on Unix, this causes failures when // the xunit runner is configured with -xml to trace out results. When it does so with // [InlineData], the parameters get written out to the results xml file, and with our // current temporary globalization implementation on Unix, this causes exceptions like // ""The surrogate pair (0xD800, 0x22) is invalid. A high surrogate character // (0xD800 - 0xDBFF) must always be paired with a low surrogate character (0xDC00 - 0xDFFF)."" ``` However, after implementing the Encoding table on Unix, and converting this test back to a Theory, we still get a similar error running the test: `error: Invalid high surrogate character (0xDFFF). A high surrogate character must have a value from range (0xD800 - 0xDBFF).` We should investigate why this error is happening and fix it. " 3538 area-Infrastructure Discussion regarding support of CoreFX builds on non-windows OSes. "Today, building CoreFX on non windows platforms uses Mono as the runtime for all the managed code we need to run (namely MSBuild and Roslyn). The long term goal is to instead run on-top of a stable CoreCLR build. However, making this change has the possibility to regress supported build platforms since CoreCLR may not run everywhere Mono does (e.g. FreeBSD, other linux Distros, etc.) As a concrete example, our tool runtime package (which we would use to run GenFacades) contains a CoreCLR which only supports Ubuntu 14.04[1](#fr1). With buildtools 97 we are at the point now where we can actually enable GenFacades and have it work on Ubuntu 14.04, but doing so will break the build on incompatible distros. The question is, do we care enough to spend engineering effort to allow the build to work on platforms that we don't have a supported CoreCLR package for? Do we expect that when MSBuild on .NET Core lands and we try to adopt it that we will remove our mono dependency entirely? What does that mean for folks like the FreeBSD port team (/cc @janhenke), who want to build our stack on FreeBSD while they bring things up if we don't have FreeBSD packages? I know that @pgavlin is chomping at the bit to get more of the build working on Linux. It would be helpful to me as well, but we need closure here before we start going making changes that might break folks. I see a few paths forward. 1. Say that building Managed CoreFX code cross platform requires your platform be supported by CoreCLR. Since today the supported platforms are Ubuntu 14.04 and OSX, update build.sh to pass the correct RID for ToolNugetRuntimeId when invoking MSBuild. If you are on a different distro and want to go out of your way to install the correct set of dependencies so an existing supported RID can run on your distro, you're free to do so and then use that RID. 2. Do platform detection and ""light up"" when we are running on a CoreCLR supported distro. In this mode, if we are able to run CoreCLR, we prefer that otherwise we fallback to Mono. For cases where we can't fallback (e.g GenFacades) we just no-op the relevant builds tasks. 3. Make the statement that managed code that runs during build time _must_ run on Mono and _may_ run on CoreCLR. This is like (2) expect it means we can always fallback. This is not true today. I'm interested in getting everyone's take on this. Maybe I'm freaking out over nothing and I can go ahead and just do (1) without anyone getting upset. 1 Technically, this could mean any OS that has a set of libraries which are binary compatible with the versions of our dependencies in Ubuntu 14.04. For some dependencies this is easy (i.e. the version of openssl that is in another distro's package manager is probably binary compatible with the version in Ubuntu 14.04, but the version of the ICU library is probably not, since they decorate their export names per version). " 3541 area-System.Net Remove Assert(asynchronous failure) from Sockets tests. In principle, any call that fails ansynchronously may also fail synchronously. 3542 area-System.Net Should sockets performance tests be inner loop? These tests have failed to meet the baseline a few times, but they pass on re-run. http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/4407/testReport/junit/System.Net.Sockets.Performance.Tests/SocketPerformanceAPMTests/SocketPerformance_SingleSocketClientAsync_LocalHostServerAPM/ When the CI System runs jobs, it can run multiple jobs on the same machine at the same time, could that be causing problems or are there real problems here that we should be investigating? It seems like the long term plan is that we move to @ianhays's [Performance] stuff. Can we just adopt these changes for that instead of having yet another perf testing thing that also runs inner loop? /cc @CIPop @davidsh @pgavlin @bartonjs. 3543 area-Infrastructure Repo organization guidelines for Common\tests Currently, there's a free-for-all regarding how code is placed in `src/Common/tests` folders, unlike the `src/Common/src` folder tree which has clear guidelines. There are a few proposals: 1) Mirror the `src/Common/src` idea and have everything properly aligned to the namespace. So, the folder paths would look like this: `src/Common/tests/System/Net/Http/HelperClass.cs`. Note that each namespace path is one word only per part in the pathname. This seems to be the most logical choice although. It could create some confusion at first since, for example, Sockets tests will add to Threading, EventSource and Resources but in the end will allow for better organization, review and increase code reuse. 2) Organize `src/Common/tests` per namespace ownership: /System.Net; /System.Xml, etc. So, the folder paths would look like this: `src/Common/tests/System.Net.Http/HelperClass.cs` This is partially what's going on today although there are folders that aren't fully specifying the namespace such as Collections, Compression, etc. Please add your opinions. 3544 area-System.Globalization "[Linux] Default locale is ""Invariant Language (Invariant Country)""" "Odd default locale with Core CLR on Linux: It is the invariant locale rather than the expected ""English (United States)"". Test just examines `Culture.CurrentCulture.EnglishName` and `Culture.CurrentUICulture.EnglishName`. Mono gets this right on the system system. I'm using `System.Globalization/4.0.11-beta-23326`, `System.Globalization.Extensions/4.0.1-beta-23326` and `System.Runtime/4.0.21-beta-23326` for most-recent tests of this scenario. " 3546 area-System.Linq Stack overflow enumerating EnumerableQuery created from null enumerable `EnumerableQuery` can be created passing null a `Expression`, an inappropriately typed `Expression`, a null `IEnumerable`, or a null value of a derived type. Blocking this would be a breaking change, especially since such objects can be usefully used as query providers. Calling `GetEnumerator()` on most such cases result in an `ArgumentNullException` or `ArgumentException`, which could probably be improved (another exception might be more appropriate), but if the constructor was passed a null `IEnumerable` then `GetEnumerator()` hits two cases where it would go into infinite recursion. First when the expression is visited, it is then recursively visited again until stack overflow. Potentially there are other ways this path could be hit, and so other bugs it could cause. If this is fixed, then `GetEnumerator()` itself will then delegate to its own `GetEnumerator()` causing infinite recursion again. ``` C# [Fact] public void NullEnumerableConstantNullExpression() { IQueryable query = new EnumerableQuery((IEnumerable)null); var exp = (ConstantExpression)query.Expression; query.GetEnumerator(); // stack overflow. } ``` 3548 area-System.Net Move Sockets performance tests into [OuterLoop]. Fixes #3495, #3542. 3550 area-System.Net Why HttpClient can't run on linux(coreclr-1.0.0-beta8-15776) "It's not ready yet? ``` C# using System; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; using System.Net.Http; namespace ConsoleApp2 { public class Program { public void Main(string[] args) { using (HttpClient client = new HttpClient()) { Task task = client.GetStringAsync(""https://github.com/""); task.Wait(); Console.WriteLine(task.Result); } Console.WriteLine(""exit!!""); } } } ``` ``` System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. File name: 'System.Net.Http, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' ---> System.IO.FileNotFoundException: Could not load the specified file. File name: 'System.Net.Http' at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyName(AssemblyName assemblyName) at System.Runtime.Loader.AssemblyLoadContext.Resolve(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName) at ConsoleApp2.Program.Main(String[] args) at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at Microsoft.Dnx.Runtime.Common.EntryPointExecutor.Execute(Assembly assembly, String[] args, IServiceProvider serviceProvider) at Microsoft.Dnx.ApplicationHost.Program.ExecuteMain(DefaultHost host, String applicationName, String[] args) at Microsoft.Dnx.ApplicationHost.Program.Main(String[] args) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Dnx.Runtime.Common.EntryPointExecutor.Execute(Assembly assembly, String[] args, IServiceProvider serviceProvider) at Microsoft.Dnx.Host.Bootstrapper.RunAsync(List`1 args, IRuntimeEnvironment env, String appBase, FrameworkName targetFramework) at Microsoft.Dnx.Host.RuntimeBootstrapper.ExecuteAsync(String[] args, BootstrapperContext bootstrapperContext) at Microsoft.Dnx.Host.RuntimeBootstrapper.Execute(String[] args, BootstrapperContext bootstrapperContext) System.IO.FileNotFoundException: Could not load the specified file. File name: 'System.Net.Http' at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyName(AssemblyName assemblyName) at System.Runtime.Loader.AssemblyLoadContext.Resolve(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName) ``` " 3554 area-System.Net Add more HTTP tests - Add more tests ported from the internal ToF tests. - Cleanup existing tests to be consistent in style and format as it has drifted due to many different people working on the tests. - Disposing the HttpClient object will dispose the handler as well so we don't need multiple 'using' statements unless we are explicitly testing dispose ordering semantics. 3555 area-System.Net System.Net.Http (Linux) allows insecure HTTPS -> HTTP redirect Discovered during PR #3554 Test failure: System.Net.Http.Functional.Tests.HttpClientHandlerTest.GetAsync_AllowAutoRedirectTrue_RedirectFromHttpsToHttp_StatusCodeRedirect MESSAGE: Assert.Equal() Failure Expected: Redirect Actual: OK +++++++++++++++++++ STACK TRACE: at System.Net.Http.Functional.Tests.HttpClientHandlerTest.d__44.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 3556 area-System.Linq Interpreting dynamic expressions failed Hello, a really Q&D example for showing the issue. First the simple callsitebinder: ``` C# private class BinaryCallSiteBinder : BinaryOperationBinder { public BinaryCallSiteBinder() : base(ExpressionType.Add) { } public override DynamicMetaObject FallbackBinaryOperation(DynamicMetaObject target, DynamicMetaObject arg, DynamicMetaObject errorSuggestion) { return new DynamicMetaObject( Expression.Convert( Expression.Add( Expression.Convert(target.Expression, typeof(int)), Expression.Convert(arg.Expression, typeof(int)) ), typeof(object)), BindingRestrictions.GetTypeRestriction(target.Expression, typeof(int)).Merge( BindingRestrictions.GetTypeRestriction(arg.Expression, typeof(int)) )); } } ``` And the calling code: ``` C# var expr = DynamicExpression.Dynamic(new BinaryCallSiteBinder(), typeof(object), Expression.Constant(40, typeof(object)), Expression.Constant(2, typeof(object))); var f = Expression.Lambda>(expr); var f2 = f.Compile(); textbox1.Text = f2().ToString(); ``` If I run the code under Windows 8.1 Phone or the desktop .net framework, it works fine. But under Windows 10 UWP (it should .net core) I get the following exception: ``` System.ArgumentException wurde nicht von Benutzercode behandelt. HResult=-2147024809 Message=Expression of type 'System.Object' cannot be used for parameter of type 'System.Runtime.CompilerServices.CallSite' of method 'Boolean SetNotMatched(System.Runtime.CompilerServices.CallSite)' Source=System.Linq.Expressions StackTrace: at System.Dynamic.Utils.ExpressionUtils.ValidateOneArgument(MethodBase method, ExpressionType nodeKind, Expression arg, ParameterInfo pi) at System.Linq.Expressions.Expression.Call(MethodInfo method, Expression arg0) at System.Runtime.CompilerServices.CallSite`1.CreateCustomNoMatchDelegate(MethodInfo invoke) at System.Runtime.CompilerServices.CallSite`1.MakeUpdateDelegate() at System.Runtime.CompilerServices.CallSite`1.GetUpdateDelegate(T& addr) at System.Runtime.CompilerServices.CallSite`1.GetUpdateDelegate() at System.Runtime.CompilerServices.CallSite`1.Create(CallSiteBinder binder) at System.Runtime.CompilerServices.CallSite.Create(Type delegateType, CallSiteBinder binder) at System.Linq.Expressions.DynamicExpression.Reduce() at System.Linq.Expressions.Interpreter.LightCompiler.CompileNoLabelPush(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.Compile(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.CompileTop(LambdaExpression node) at System.Linq.Expressions.Expression`1.Compile() at TestLua.MainPage.btnClick_Click(Object sender, RoutedEventArgs e) ``` What is wrong? 3557 area-System.Net System.Net.Http (Linux) not passing credentials (in cache) during redirect Discovered during PR #3554 Test Failure: System.Net.Http.Functional.Tests.HttpClientHandlerTest.GetAsync_CredentialIsCredentialCacheUriRedirect_StatusCodeOK MESSAGE: Assert.Equal() Failure Expected: OK Actual: Unauthorized +++++++++++++++++++ STACK TRACE: at System.Net.Http.Functional.Tests.HttpClientHandlerTest.d__5c.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 3558 area-System.Console Expose System.Console.WindowWidth Azure CLI would like us to expose WindowWidth property 3559 area-System.Console Expose System.Console.CursorVisible Azure CLI would like us to expose this property 3562 area-System.Net Tracking issues for System.Net.NetworkInformation and System.Net.Utilities This issue is tracking TODOs and Shims removal for PR #: - [ ] Replace Interop.\* with OS names from the SDK (iphlpapi) - [ ] Unify error codes for Win32 (see Interop.ErrorCodes.cs in iphlpapi) - [ ] Investigate DangerousAddRef then Dispose within `SafeCloseSocketAndEvent::CompleteInitialization(...)` - [ ] In NetworkAddressChange.cs : > @stephentoub: We're invoking the user's callback while holding s_syncObject. There are trivial ways that could lead to deadlock (e.g. if the callback queued and waited on a work item that tried to add a handler), though maybe we don't care about such degenerate patterns? - [ ] In NetworkAddressChange.cs: StartHelper() > @ stephentoub: Is it not possible for both IPv4 and IPv6 to be supported and in the start options? If it is possible, we're overwriting statics that were just set in the previous section. > > > @CIPop Collaborator > > CIPop added a note just now > > Indeed, s_registeredWait is overwritten. There are other issues with this implementation: the RegisteredWaitHandle is never unregistered. https://msdn.microsoft.com/en-us/library/w9f75h7a(v=vs.110).aspx: > > > When you are finished using the RegisteredWaitHandle that is returned by this method, call its RegisteredWaitHandle.Unregister method to release references to the wait handle. We recommend that you always call the RegisteredWaitHandle.Unregister method, even if you specify true for executeOnlyOnce 3563 area-System.Collections Add System.Collections.Concurrent.ConcurrentHashSet "We already have classes in the `System.Collections.Concurrent` namespace that provide (mostly) equivalent APIs to most of those in the `System.Collections.Generic` namespace. However, one omission is a concurrent equivalent for `System.Collections.Generic.HashSet`. At the moment, if a thread-safe hash set is needed, we are left with one of three options: - Creating a thread-safe collection class from scratch. - Synchronizing `System.Collections.Generic.HashSet` in a wrapper class. - Using `System.Collections.Concurrent.ConcurrentDictionary` with ""dummy"" values such as `object` or `byte`, either directly or through a wrapper. None of these approaches is ideal. The first and second option are both error prone (concurrency is hard) and difficult to fully optimize. The third wastes memory and time spent allocating the dummy objects. It seems reasonable to expect such a class wouldn't be too challenging to create given that it would appear to mostly consist of reducing the functionality in the already-existing `ConcurrentDictionary` class (though that one is ~2,000 line long, so...). " 3564 area-System.Net Add unidirectional tests for Socket.{Send,Receive}. These tests send data in a loop from one socket to another and check that all data was correctly received. There are tests for each of: - {SendTo,ReceiveFrom} - {Begin,End}{SendTo,ReceiveFrom} - {SendTo,ReceiveFrom}Async - {Send,Receive} - {Begin,End}{Send,Receive} - {Send,Receive}Async The SendTo/ReceiveFrom tests use UDP sockets; the others use TCP sockets. All tests use either the IPv4 or IPv6 loopback interface. 3565 area-System.Net System.Net.Http (Linux) doesn't support ChannelBinding Test (HttpClientHandlerTest.cs) PostAsync_Post_ChannelBindingHasExpectedValue() fails because it returns a NULL value for ChannelBinding. 3566 area-System.Runtime Perf: Environment.ExpandEnvironmentVariables 4x slower on Linux "Perf test: ``` PerfUtils utils = new PerfUtils(); string env = utils.CreateString(15); string inputEnv = ""%"" + env + ""%""; try { // setup the environment variable so we can read it Environment.SetEnvironmentVariable(env, ""value""); // read the valid environment variable foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i < 40000; i++) { Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); Environment.ExpandEnvironmentVariables(inputEnv); } } finally { // clear the variable that we set Environment.SetEnvironmentVariable(env, null); } ``` Linux results: ``` ``` Windows results: ``` ``` " 3568 area-System.Runtime Perf: Environment.GetEnvironmentVariable 4x slower on Linux "Perf test: ``` [Benchmark] public void GetEnvironmentVariable() { PerfUtils utils = new PerfUtils(); string env = utils.CreateString(15); try { // setup the environment variable so we can read it Environment.SetEnvironmentVariable(env, ""value""); // read the valid environment variable for the test foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i < 40000; i++) { Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); Environment.GetEnvironmentVariable(env); } } finally { // clear the variable that we set Environment.SetEnvironmentVariable(env, null); } } ``` Linux results: ``` ``` Windows results: ``` ``` " 3569 area-System.Runtime System.Runtime.Loader.Tests.AssemblyLoadContextTest.InitializeDefaultContextTest failing on Linux in CI For example: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_linux_debug_tst_prtest/2896/ 3570 area-System.Net Fix Net.NameResolutionPal.Unix when sockaddr shim'ing work comes in The System.Net.NameResolutionPal.Unix::TryGetAddrInfo function does a lot of copying a bit twiddling according to sockaddr sizes per-platform. This should be cleaned up when the sockaddr struct is shim'd 3571 area-System.Runtime Perf: DateTime.Now 3x slower on Linux "Perf test: ```c# [Benchmark] public void GetNow() { DateTime dt; foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i < 10000; i++) { dt = DateTime.Now; dt = DateTime.Now; dt = DateTime.Now; dt = DateTime.Now; dt = DateTime.Now; dt = DateTime.Now; dt = DateTime.Now; dt = DateTime.Now; dt = DateTime.Now; } } ``` Linux results: ```xml ``` Windows results: ```xml ``` " 3572 area-System.Runtime Perf: new Random() 2x slower on Linux "Perf test: ``` [Benchmark] public void ctor() { foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i < 40000; i++) { new Random(); new Random(); new Random(); new Random(); new Random(); new Random(); new Random(); new Random(); new Random(); } } ``` Linux results: ``` ``` Windows results: ``` ``` I expect this is due to the use of DateTime.Now as the seed for the default Random constructor, making this related to #3571 " 3573 area-System.Runtime Perf: Guid.NewGuid 200x slower on Linux "Perf test: ``` [Benchmark] public void NewGuid() { foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i < 10000; i++) { Guid.NewGuid(); Guid.NewGuid(); Guid.NewGuid(); Guid.NewGuid(); Guid.NewGuid(); Guid.NewGuid(); Guid.NewGuid(); Guid.NewGuid(); Guid.NewGuid(); } } ``` Linux results: ``` ``` Windows results: ``` ``` " 3574 area-System.Runtime Perf: String.ToUpper and String.ToLower 3x slower on Linux "Perf test and relevant helper methods: ``` [Benchmark] [MemberData(""TestStringSizes"")] public void ToUpper(int size) { PerfUtils utils = new PerfUtils(); string testString = utils.CreateString(size); foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i < 10000; i++) testString.ToUpper(); } public static IEnumerable TestStringSizes() { yield return new object[] { 10 }; yield return new object[] { 100 }; yield return new object[] { 1000 }; } /// /// Helper method to create a string containing a number of random /// characters equal to the specified length /// public string CreateString(int length) { byte[] bytes = new byte[length]; rand.NextBytes(bytes); return System.Convert.ToBase64String(bytes); } ``` Linux results (same for ToLower) for length=1000: ``` ``` Windows results (same for ToUpper) for length=1000: ``` ``` Results for other lengths are similar but less severe. I have excluded them for the conciseness but may post them if desired. " 3575 area-System.Net Fix a bug in Windows async sockets. Consider the following sequence of events for a SocketAsyncEventArgs `args` and a non-null `byte` buffer `buf`: 1. `args._buffer` is set to `buf` via `args.SetBuffer(buf, 0, buf.Length)` 2. Some asychronous call is made using `args` (e.g. `socket.SendAsync(args)`) 3. The asynchronous call completed 4. `args._buffer` is set to `null` via `args.SetBuffer(null, 0, 0)` 5. A GC occurs that relocates `buf`. 6. `args._buffer` is reset to `buf` via `args.SetBuffer(buf, 0, buf.Length)` 7. Another asynchronous call is made using `args` (e.g. `socket.SendAsync(args)`) - (2) will pin `buf` via a `PreAllocatedOverlapped` instance, store the instance in `args._preAllocatedOverlapped`, set `args._pinState` to `PinState.SingleBuffer`, set up a native overlapped object, and finally set `args._ptrNativeOverlapped` to a safe handle that wraps the overlapped object. - (3) will dispose the handle stored in `args._ptrNativeOverlapped` and set `args._ptrNativeOverlapped` to `null`. - (4) will call `args.FreeOverlapped`, which will dispose `args._preAllocatedOverlapped` (thus unpinning `buf`) and set `args._preAllocatedOverlapped` to `null`. `args._pinState` and related fields will not be updated: the code that does so is conditional upon `args._ptrNativeOverlapped` being non-`null`, and (3) already set the same to `null`. - (6) will call `args.CheckPinSingleBuffer`, which will observe that `args._pinState` is `PinState.SingleBuffer` and `_pinnedSingleBuffer` is the same as `_buffer` and consider the buffer already pinned. As a result, the call will not re-pin `buf`. - Because (6) did not re-pin `buf`, the asynchronous call made in (7) will reuse the address at which the buffer was located the last time it was pinned. At best, this address is no longer mapped and the call results in an AV; at worst, this address contains unknown (and potentially sensitive) data. The fix is to reset `_pinState` and related fields when `_preAllocatedOverlapped` is disposed. 3576 area-Infrastructure Is there a Resharper settings file for this coding style? I personally dislike the default R# C# coding style configuration. It would be nice to have R# configured for this coding style. Then I could use R#'s code reformatting functionality to bring code into line. :-) While the stand-alone CodeFormatter.exe is nice to have, for the rest of my team, if it isn't integrated into their VS workflow it might as well not exist. 3578 area-System.Net CURL Handler redirect fixes This commit fixes the following issues in CurlHandler w.r.t following URI redirection - Ensure that the credentials are nullified at the right place. We should not set username and/or password to null unnecessarily. - Ensure that libcurl follows only HTTPS uri if the original request was for a HTTPS URL. - Ensure that libcurl follows only a HTTPS or a HTTP uri if the original request was for a HTTP URL. 3580 area-System.Net NullReferenceException in System.Net.Sockets.Tests.Disconnect.Success on Windows http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/4466/console ``` C# Discovering: System.Net.Sockets.AsyncServer.Tests Discovered: System.Net.Sockets.AsyncServer.Tests Starting: System.Net.Sockets.AsyncServer.Tests System.Net.Sockets.Tests.Disconnect.Success [FAIL] System.NullReferenceException : Object reference not set to an instance of an object. Stack Trace: d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets\src\System\Net\Sockets\Socket.Windows.cs(86,0): at System.Net.Sockets.Socket.DisconnectExBlocking(IntPtr socketHandle, IntPtr overlapped, Int32 flags, Int32 reserved) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets\src\System\Net\Sockets\SocketPal.Windows.cs(156,0): at System.Net.Sockets.SocketPal.Disconnect(Socket socket, SafeCloseSocket handle, Boolean reuseSocket) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets\src\System\Net\Sockets\Socket.cs(2600,0): at System.Net.Sockets.Socket.Disconnect(Boolean reuseSocket) d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Net.Sockets.Legacy\tests\FunctionalTests\Disconnect.cs(42,0): at System.Net.Sockets.Tests.Disconnect.Success() Finished: System.Net.Sockets.AsyncServer.Tests ``` cc: @davidsh, @cipop, @pgavlin 3583 area-System.Threading Timer_Change_BeforeDueTime_ChangesWhenTimerWillFire test fails A timer test failed on PR here: http://dotnet-ci.cloudapp.net/job/dotnet_corefx_windows_debug_prtest/4494/testReport/junit/(root)/TimerChangeTests/Timer_Change_BeforeDueTime_ChangesWhenTimerWillFire/ MESSAGE: Should have received a timer event after this new duration Expected: True Actual: False +++++++++++++++++++ STACK TRACE: at TimerChangeTests.Timer_Change_BeforeDueTime_ChangesWhenTimerWillFire() in d:\j\workspace\dotnet_corefx_windows_debug_prtest\src\System.Threading.Timer\tests\TimerChangeTests.cs:line 94 3588 area-System.Collections Perf: System.Collections List, Dictionary default constructors 2x slower on Linux "Perf Tests: ``` [Benchmark] public void ctor() { foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) for (int i = 0; i <= 20000; i++) { new Dictionary(); new Dictionary(); new Dictionary(); new Dictionary(); new Dictionary(); new Dictionary(); new Dictionary(); new Dictionary(); new Dictionary(); } } [Benchmark] public void ctor() { foreach (var iteration in Benchmark.Iterations) using (iteration.StartMeasurement()) { for (int i = 0; i < 20000; i++) { new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); new List(); } } } ``` Linux perf results for Dictionary: ``` ``` Windows perf results for Dictionary: ``` ``` Linux perf results for List: ```