= Troubleshoot Policy Enforcement Issues
ifndef::env-site,env-github[]
include::_attributes.adoc[]
endif::[]
:keywords: service mesh, microservices, policy enforcement issues, troubleshooting

Even though you applied policies to your services, the Mule adapter might not seem to be enforcing these policies, responding with the following error messages:

* <<service-port-not-named,Service port is not named>>
* <<policy-not-disassociated-after-deleting-bindings,Policy not disassociated after deleting bindings>>
* <<binding-service-name-does-not-match-actual-service-name,Binding service name does not match actual service name>>
* <<app-pod-does-not-have-envoy-sidecar,Application pod does not have an Envoy sidecar>>
* <<internal-error-with-500-error-code,Internal error with 500 error code>>


[[service-port-not-named]]
== Service Port Is Not Named

When you create a service, you must provide specifications, such as ports and selectors. Each of these has its own set of parameters that must be defined. When you assign a port to a service, you must also specify a name for that port, such as "http," to avoid an error. Istio requires you to link:https://istio.io/docs/ops/deployment/requirements/[name every port].

=== Causes

A name for the service port is not specified.

=== Diagnose

To diagnose this issue, verify the definition of the service that exposes the application:

`$ kubectl -n _namespace_ get services -o yaml`

For example, the following service does not have a named port:

[source,text,linenums]
----
apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    creationTimestamp: "2019-10-09T19:08:28Z"
    name: hello-kubernetes
    namespace: default
    resourceVersion: "7075"
    selfLink: /api/v1/namespaces/default/services/hello-kubernetes
    uid: 2d0719a5-eac8-11e9-93c9-06b247319cf6
  spec:
    clusterIP: 172.20.177.73
    externalTrafficPolicy: Cluster
    ports:
    - nodePort: 32231
      port: 80
      protocol: TCP
      targetPort: 8080
    selector:
      app: hello-kubernetes
    sessionAffinity: None
    type: LoadBalancer
----

=== Solution

Istio requires you to link:https://istio.io/docs/ops/deployment/requirements/[name every port]. To resolve this error, re-create the service with a named port:

[source,text,linenums]
----
apiVersion: v1
kind: Service
metadata:
  name: hello-kubernetes
spec:
  type: LoadBalancer
  ports:
  - port: 80
    name: http
    targetPort: 8080
  selector:
    app: hello-kubernetes
----

[[policy-not-disassociated-after-deleting-bindings]]
== Policy Not Disassociated after Deleting Bindings

After you delete the bindings, the policies applied to the service might not be successfully removed.

== Causes

When a binding was deleted, an associated traffic rule was not deleted. Because of this issue, traffic is still being sent to the Mule adapter.

=== Diagnose

To diagnose this issue, verify that the service rules have been deleted:

`$ kubectl get rules -l serviceName=_service name_`

If you determine that the rules are still applied, manually delete those rules.

=== Solution

Delete the incorrect rules applied to the service:

`$ kubectl delete rules -l serviceName=_service_name_`


[[binding-service-name-does-not-match-actual-service-name]]
== Binding Service Name Does Not Match Actual Service Name

When you create the binding, you must provide a service name. If the service name provided does not match the actual service name, policies are not enforced.

=== Causes

Binding was created pointing to an nonexistent service.


=== Diagnose

To diagnose this issue:

. Review the service bindings using either of the following commands:
+
* `$ asmctl api binding list`
* `$ asmctl adapter binding list` 
. Compare the service with the bindings:
+
`$ kubectl -n <namespace> get services`
+
image::binding-service-does-not-match-get-services-service-mesh.png[Binding service does match actual service error]

In the above example, service names must be, for example `customer-service`, and NOT `customers.nto-payment.svc.cluster.nto-cluster`

=== Solution

To resolve this issue:

. Delete the existing bindings by running any of the commands:
+
* `$ asmctl api binding delete --name=_binding_name_ --namespace=_binding_namespace_`
* `$ asmctl adapter binding delete --name=_binding_name_ --namespace=_binding_namespace_`
+
. Re-create the bindings to point to the actual services:

image::binding-service-does-not-match-solution-service-mesh.png[Binding service does not match actual service resolution]

[[app-pod-does-not-have-envoy-sidecar]]
== Application Pod Does Not Have an Envoy Sidecar

Policies are not enforced due to an Envoy error in the application pod.

== Causes

Applications were not restarted in the namespace that was marked for injecting Envoy sidecar for every pod.

=== Diagnose

To diagnose this issue, verify that the sidecar has an envoy proxy associated to it:

`$ asmctl management check sidecar`
 

=== Solution

If the terminal does not display the status as `Injected`, you must restart the pods: 

`$ kubectl  -n <namespace> delete pods <pod_name>`

[[internal-error-with-500-error-code]]
== Internal Error with 500 Error Code

The applications managed by Anypoint Service Mesh generate a 500 error code with the following error description: 

`INTERNAL:grpcmule-nto-payment.handler.service-mesh`.`INTERNAL:grpcmule-nto-payment.handler.service-mesh`

=== Causes

This error occurs if the license expires and, as a result, the adapter restarts.

=== Diagnose

To diagnose this issue:

. Display the adapter pod:
+
`$ kubectl get pods -n service-mesh`
. Retrieve the adapter logs:
+
`$ kubectl -n service-mesh logs $(kubectl -n service-mesh get pod -l namespace=_replace_with_namespace_ -oname) -c mule`
+
If the license has expired, an error similar to the following is displayed:
+
image::internal-500-error-msg-service-mesh.png[Internal 500 error message]

=== Solution

To resolve this issue: 

. Replace the existing License with a valid one:

+
`$ asmctl management config license --license=/path/to/license.lic`

. Redeploy the adapter:
+
`$ kubectl -n service-mesh patch $(kubectl -n service-mesh get deploy -l namespace=_replace_with_namespace_ -oname) --type=json -p="[{\"op\": \"replace\", \"path\": \"/spec/template/metadata/annotations/kubectl.kubernetes.io~1restartedAt\",\"value\":\"$(date -u +%FT%TZ)\"}]"`