Problem description

Use Azure virtual network ,Azure APIM Can manage, can't pass Internet Access to the API, To protect the back end within the enterprise API Purpose . In a virtual network , Enable network security groups (NSG:Network Security Group) To control the outbound , The inbound port , We can compare it to a door , The traffic that needs to meet the conditions can pass through this safe gate .

After passing through the security gate , We need a way , To achieve the goal . And the default network path is Azure Backbone network control of , If you need to go through the company's own firewall (Azure Firewall Or the enterprise's own firewall ). In a virtual network , Routing table (Route Table) To specify the next hop (Next Pod), By using 0.0.0.0/0 Configuration mode (UDRs:user-defined routes), All the traffic in the virtual network is imported and passed through the firewall . Achieve enterprise level security .

To sum up, the above two sentences are :NSG It's a door. ,UDR It's the way . The solution architecture diagram is :

however , In the building APIM VNET When , But met all kinds of mistakes .

1) because NSG Why , The required port is not working . Lead to APIM Network connection status of “Network connectivity status” The status in the page is Error

2) Because the custom route is configured (UDR), Led to APIM On the page APIs,Repository Wait for page error , And the portal keeps popping up Failed to connect to management endpoint Error message

Problem analysis

First : The above two questions , Can be in "  How to integrate Azure API Management works with virtual networks : Common network configuration problems " Find the answer in the article .

Problem one, port problem : Add all the necessary ports listed in the above document , Can realize the network connection state all Success The state of .

API Manage the required ports : have access to   Network security group Control of the deployment of API Inbound and outbound traffic of managed subnet .  If either port is not available ,API Management may not work properly and may not be accessible .  take API Management and VNET When used in combination , Another common misconfiguration problem is blocking one or more of these ports .

VNET Middle trusteeship APIM Service instance , The ports in the table below will be used .

Source / Target port Direction Transfer protocol Service mark
Source / The goal is
Purpose (*) Virtual network type
* / [80], 443 Inbound TCP INTERNET/VIRTUAL_NETWORK The client and API Managed Communications external
* / 3443 Inbound TCP ApiManagement / VIRTUAL_NETWORK Azure Portals and PowerShell The end point of Management Outside and inside
* / 443 Departure TCP VIRTUAL_NETWORK / Storage And Azure Storage dependencies Outside and inside
* / 443 Departure TCP VIRTUAL_NETWORK / AzureActiveDirectory Azure Active Directory  and Azure KeyVault Dependencies Outside and inside
* / 1433 Departure TCP VIRTUAL_NETWORK / SQL visit Azure SQL endpoints Outside and inside
* / 443 Departure TCP VIRTUAL_NETWORK / AzureKeyVault visit Azure KeyVault Outside and inside
* / 5671, 5672, 443 Departure TCP VIRTUAL_NETWORK / EventHub Event center policy log And monitoring agent dependencies Outside and inside
* / 445 Departure TCP VIRTUAL_NETWORK / Storage And apply to  GIT  Of Azure File sharing dependencies Outside and inside
* / 443, 12000 Departure TCP VIRTUAL_NETWORK / AzureCloud Health and monitoring extensions Outside and inside
* / 1886、443 Departure TCP VIRTUAL_NETWORK / AzureMonitor Release Diagnostic logs and indicators Resource health and  Application Insights Outside and inside
* / 25、587、25028 Departure TCP VIRTUAL_NETWORK/INTERNET Connect to SMTP Relay to send email Outside and inside
* / 6381 - 6383 Inbound and outbound TCP VIRTUAL_NETWORK/VIRTUAL_NETWORK visit Redis Service to get information between computers cache Strategy Outside and inside
* / 4290 Inbound and outbound UDP VIRTUAL_NETWORK/VIRTUAL_NETWORK Synchronization is used between computers Rate limit Counter for policy Outside and inside
* / * Inbound TCP AZURE_LOAD_BALANCER/VIRTUAL_NETWORK Azure Infrastructure load balancer Outside and inside

Configuration complete NSG after , Remember that , stay APIM In the middle Apply Network Configration, To achieve bilateral synchronization of network configuration .

Problem 2: Custom routing problem : It's more complicated . The service endpoint needs to be opened , Release... In China in routing table IP Address ( Control plane IP Address

Use Express Route Or network virtual devices force tunnel traffic to the local firewall : A common configuration for customers is to define their own default routes (0.0.0.0/0), Coercion comes from API Manage all the traffic of the entrusted subnet to flow through the local firewall or to the network virtual device .  This flow is bound to be interrupted with Azure API Managed connections , Because outbound traffic will be blocked locally , Or by “ Network address translation ” Functions sent to are no longer associated with various Azure A set of unrecognized addresses that endpoints work with .

This solution requires three actions :

The first one is : In the deployment API Enable service endpoints on subnets of management services .  Need to be Azure SQL、Azure Storage 、Azure The event center and Azure Service bus enabled Service endpoint .  Directly from API Managing the subnet of the delegate enables the endpoints for these services to use Azure Backbone network , Provide optimized routing for service traffic .  If you combine a service endpoint with a forced Tunnel API Management with the use of , The above will not be Azure Service traffic is forced to Tunnel .  other API Manage service dependency traffic through forced tunnel redirection , You can't lose , otherwise API Management services can be dysfunctional .

The second item : All control plane flow ( from Internet To API The management endpoint of management services ) It's going to go through a specific set of API Manage managed inbound IP Routing .  When traffic is forced to Tunnel , The response does not map symmetrically back to these inbound sources IP.  To overcome this limitation , We need to add the following user-defined routes (UDR), By setting the destination of these host routes to “Internet” To send traffic back to Azure.  Inbound to control plane traffic IP The collection is on record Control plane IP Address

The third one : For others forced to Tunnel API Manage service dependencies , There should be a way to resolve the host name and access the endpoint .  These include :

  • Indicators and health monitoring : To Azure Monitor the outbound network connection of the endpoint , It can be parsed in the intranet , these URL stay AzureMonitor Under the service tag, it means , For network security groups .
        1. mooncake.warmpath.chinacloudapi.cn
        2. global.prod.microsoftmetrics.com( newly added )
        3. shoebox2.prod.microsoftmetrics.com( newly added )
        4. shoebox2-red.prod.microsoftmetrics.com
        5. shoebox2-black.prod.microsoftmetrics.com
        6. shoebox2-red.shoebox2.metrics.nsatc.net
        7. shoebox2-black.shoebox2.metrics.nsatc.net
        8. prod3.prod.microsoftmetrics.com( newly added )
        9. prod3-red.prod.microsoftmetrics.com
        10. prod5.prod.microsoftmetrics.com
        11. prod5-black.prod.microsoftmetrics.com
        12. prod5-red.prod.microsoftmetrics.com
        13. gcs.prod.warm.ingestion.monitoring.azure.cn
  • Azure Portal diagnostics : To use from within the virtual network API Manage extensions from Azure Portal enables diagnostic log flow , Need to be allowed on Port 443 Yes  dc.services.visualstudio.com  Make an outbound visit .
  • SMTP relay   : Host computer  smtpi-co1.msn.comsmtpi-ch1.msn.comsmtpi-db3.msn.comsmtpi-sin.msn.comies.global.microsoft.com  I'm going to analyze it SMTP Outbound network connection for relay
  • Developer portal captcha : Host computer  client.hip.live.com  and  partner.hip.live.com  The developer portal that is parsed under CAPTCHA Outbound network connection for .

Problem solving

contrast NSG list , Add missing ports . especially :1433,5671, 5672,12000,1886,25028, 6381 - 6383 And so on .

stay Route Table Add two Chinese regions marked as global IP Address . This is the required address , It is also a problem that is often ignored in configuration . It's also the root of the problem .

Reference material

Use in an internal virtual network Azure API management service :https://docs.azure.cn/zh-cn/api-management/api-management-using-with-internal-vnet

APIM Common network configuration problems :https://docs.azure.cn/zh-cn/api-management/api-management-using-with-vnet#common-network-configuration-issues

【Azure API management 】APIM After integrating the intranet and virtual network , Enable custom routing to manage outgoing traffic through the firewall (Firewall), meet APIs More related articles that can't load out problems

  1. 【Azure Application service 】App Service And APIM After integrating into the same virtual network at the same time , How to access the internal network through the intranet VNET Of APIM Well ?

    Problem description App Service Access to the APIM Configured internal virtual network (Internal VNet) And have an intranet IP Address .App Service And APIM All in the same virtual network (VNET) in .App Servic ...

  2. The way of Cloud Computing - The trial Azure: Build your own intranet DNS The server

    We wrote a blog about Azure Built in Intranet DNS The server cannot span Cloud Service, And our virtual machine deployment scenario just needs to span multiple Cloud Service, So at present, we can only choose to use Azure Virtual machines build their own internal ...

  3. VMware Next CentOS7 After installation , After restoring the virtual network , knock ifconfig Don't show lan ip resolvent

    VMware Next CentOS7 After installation , After restoring the virtual network , knock ifconfig Don't show lan ip, There was no eth0 network card , Have no access to the Internet ,SSH Can't connect , Input ifconfig As shown in the following figure : resolvent : 1. Edit the configuration file of network card ...

  4. WinServer And After the intranet publishes the website, the port maps the Internet access

    Intranet IP The connection can only be accessed on the intranet , You can't know the intranet on the Internet IP Inaccessible . If you have routing rights , And the route has a fixed public network IP, Port mapping through routing , Access the intranet from the Internet . If there is no route , Or route without public network IP, Need to use a third party open peanut ...

  5. Typical case collection - Use OpenVPN Connect multiple computer room intranet ( turn )( Static routing )

    explain : 1. This article mainly uses static routing table to realize multiple computer rooms through VPN Interworking of subnetwork computer rooms after connection . 2.OpenVPN Using bridge mode (server-bridge and dev tap), This is the key point , Only in this way can ...

  6. be based on frp An example of Intranet penetration of 2- Access to intranet deployed through a custom domain name web service

    Original address :https://wuter.cn/1837.html/ One . What you want to achieve 1. Use the website deployed on your computer for public access . 2. Resolve the unregistered domain name to the domestic server ( That is, the old hen in my dormitory ). Two . Server configuration ...

  7. TP5 After the verification code is uploaded to the Alibaba cloud Wan virtual host , Verification code does not show the solution

    TP5 No captcha Just clear the buffer , Today I just met , The solution is vendor/topthink/think-captcha/CaptchaController.php Add this ob_cl ...

  8. 【API management APIM】APIM Integrated internal VNet when , Often met about custom DNS Service issues .

    Problem description Azure Of APIM There are two ways to integrate virtual networks , external VNET, Inside VNET. external VNET, Low requirements , Can pass APIM visit VNET Medium VM And so on , No need to configure custom DNS The server , In this way ,AP ...

  9. 【Azure Redis cache Azure Cache For Redis】 In creating the advanced layer Redis(P1) Integrated virtual network (VNET) after , How to test VNET How to successfully access resources and configure the effect of whitelist

    When using Azure Redis In the advanced version , For better protection Redis The safety of the , Virtual network enabled , hold Redis Integrated into the Azure The virtual network in , Only through the virtual network VENT Access to resources in , The public network is not accessible . however ...

  10. The way of Cloud Computing - The trial Azure: How to establish intranet connection between virtual machines

    On aliyun , All virtual machines created by the same account ( Cloud server ) The intranet between them is directly connected . and Azure It's totally different , I was a little overwhelmed at first , And then I found out —— stay Azure Only in the same virtual network (Virtual N ...

Random recommendation

  1. [Erlang 0103] Erlang Resources Information station

        I haven't blog for a long time , Are you lazy ? No ; In the first two months, I was pushing forward a project , There's a little accumulated in the middle RabbitMQ and Erlang Things that are ; I'm going to summarize after the project , As a result, my wife's life was in a mess after she became pregnant : Get up early and cook , evening ...

  2. openjdk Complete compilation guide

    from openjdk.java.net download openjdk Software package , You get all the relevant source code . It is strongly recommended to read carefully first  README-builds.html guide . In execution make all Before , First of all Of board ...

  3. CSS Skills summary , Research and practice

    I've been studying CSS, Because I found that I spent most of my time writing CSS, And I feel that I write poorly , Although I used to watch a lot , But it's rarely practiced , Not to mention research , Now it turns out that you don't understand at all, you will , A lot of them are used by you, and then you can really use them , So now ...

  4. windows The configuration Faster-RCNN

    mark One http://yun.baidu.com/share/link?shareid=1018944597&uk=1543560377 http://blog.csdn.net/sin ...

  5. Vision SLAM The foundation of mathematics in Fourth articles Lie groups and Lie algebras (2)

    Preface Understand Lie groups and Lie algebras , It's understanding a lot of SLAM The basis of the key issues in the project . In this lecture, we will continue to introduce the related knowledge of Lie groups and Lie algebras , Focus on the calculus of Lie groups and Lie algebras , It is of great significance to solve the problem of attitude estimation . review To describe motion in three dimensions , We ...

  6. Distributed cache memcache Study

    1. The purpose of using distributed cache is to solve the problem of multiple machines sharing information , By visiting a ip And ports to access different IIS The server 2.memcache Basic principle stay Socket Server side storage data is stored in the form of key value pairs Memory processing algorithm ...

  7. Standard type String( I am learning )

    1. Read string object #include<iostream> #include<cstring> using namespace std; int main() { stri ...

  8. css Public style , initialization

    /* CSS Document */ body, button, select, textarea, input, label, option, fieldset, legend{font-famil ...

  9. Spring frame Detailed explanation ( One )

    Spring It's layered JavaSE/EE full-stack( "One-stop" work style )  Lightweight open source framework *  layered : * SUN Provided EE The three-tier structure of :web layer . The business layer . Data access layer ( Persistence layer , Integration layer ) * Struts2 yes ...

  10. jQuery Realization The browser goes back to where it was last viewed

    Recently, Tencent . Sina's mobile website , I found a very painful thing , Browse the content in the list , I turn down , Flip down , All of a sudden , See a very domineering title , So click on the article to see the details , Some time later , Click the back button , The browser went back to the top of the page . On ...