Cisco Meraki: A Day In The Life

Wi-Fi trouble ticket complaints have been the same since the beginning of Wi-Fi.It's not working. It's dropping, the Wi-Fi is down and the end user only knows how to explain things so far. They give you inaccurate data because they don't know what you need to know.

The person who's receiving the complaint needs to know what type of questions to ask to get at The Who, what, where, when, and find out the why.

Information gathering:

The first thing you need to know before you can troubleshoot is are they using a Mac or are they using Windows?Depending on which one they're on, you will use different command-line tools to get information about the end user's machine.

Doing a DNS test will tell you whether the website is reachable for everyone. Or is the problem actually not the Wi-Fi?If you're doing a DNS test, you're testing whether or not you can resolve a website name to an IP address. If the client device can't resolve a website name to an IP address, it's not a wireless problem; it's a DNS problem.

So additional questions, what's the machine name is the problem with a specific application or all applications?Which applications is the problem with their machine?

Are five or more people experiencing the same issue?

If you look them up in the Meraki dashboard, you'll need their machine name. If it's a particular application that's having issues, that question leads to going down the rabbit hole of which application? Does that application have latency issues? Is that application server overloaded? Do you need to bring in that application server tech team to look at their server?

If it's a problem with more five or more people, it may be a larger wireless issue or it could be a larger issue with DNS or DHCP, or that ffice's WAN connectivity.

To perform extra communication testing, you will need to know the client's IP address or MAC address. Ping, traceroute and nslookup

If there's an IP conflict or duplicate IP address issue, you can look at the access point logs. These are not wireless issues, but looking in the logs may give you an indicator of where the problem lies.

What is their OS type? Knowing the operating system (OS) of a Wi-Fi client's computer is crucial when troubleshooting because different OSes handle Wi-Fi connectivity, network configuration, and troubleshooting tools in unique ways.

Here's why the client's OS is important:

Different Network Management Tools: Each OS has its own set of network management tools and commands for checking connectivity, configuring Wi-Fi, and troubleshooting network issues. For instance:

macOS uses tools like networksetup, scutil, and GUI options in System Preferences.
Windows uses the Network & Internet settings, ipconfig, netsh, and ping commands.
Linux distros typically use tools like nmcli (NetworkManager), iwconfig, and the ifconfig or ip command.
The troubleshooting steps often differ significantly based on which tools are available on the client's OS.

OS-Specific Wi-Fi Behavior and Settings: Different OSes manage Wi-Fi connections differently, affecting aspects like:

Roaming: Windows and macOS handle roaming between access points differently, and certain settings (like Preferred Network Profiles on Windows) may impact connection stability.

Power Management: Windows, for instance, has power-saving features that can turn off Wi-Fi adapters to conserve battery, potentially causing intermittent connectivity issues.

Wi-Fi Profile Management: In macOS, Wi-Fi networks can be reordered in priority, while in Windows, you may need to delete old or conflicting Wi-Fi profiles using specific commands.

Troubleshooting Compatibility Issues: Older OSes or certain OS versions may lack support for modern Wi-Fi standards, such as WPA3 security, 5 GHz bands, or certain authentication protocols. Knowing the OS helps determine if the client's device supports the security protocols and frequency bands required by the network.

Network Adapter Drivers and Updates: Windows and Linux OSes, in particular, rely heavily on drivers for Wi-Fi adapters, which can vary by version and require regular updates to fix bugs or improve performance. Outdated drivers on these OSes can lead to connectivity issues. In contrast, macOS has its drivers built into the system, with updates delivered through system updates.

Device-Specific Issues and Known Bugs: Different OS versions may have specific, known Wi-Fi issues or bugs. For example:

Windows 10 and 11 have had updates that temporarily caused Wi-Fi issues for some users.
macOS sometimes has known issues related to specific macOS updates that affect Wi-Fi stability.
Linux distributions often require kernel-specific fixes for certain Wi-Fi chipsets.

Unique Security and Firewall Configurations: Each OS has distinct security settings and firewalls that might affect Wi-Fi connectivity. Windows Defender Firewall, macOS's built-in firewall, and iptables on Linux may block network traffic or DNS queries, impacting connectivity. Knowing the OS allows you to check and modify these settings as needed.

Guidance for the User: Depending on the OS, the guidance you provide for accessing network settings, running command-line tests, or collecting diagnostic logs will differ. OS-specific instructions make it easier for the client to follow your troubleshooting steps, reducing confusion and helping achieve faster results.

Client IP address & Client MAC address: Obtaining a Wi-Fi client's IP and MAC address is essential for effective troubleshooting because these identifiers help you track and diagnose network connectivity issues at various layers:

Network Identification (MAC Address): The MAC address is the unique identifier of the Wi-Fi client's network interface. It is vital for identifying the client at the link layer (Layer 2) on the network. Network devices like access points (APs) use the MAC address to communicate with specific clients, and administrators use it to track clients across the network, check for connectivity issues, or identify whether the client is associated with the correct AP.

IP Address for Network Layer Connectivity: The IP address is necessary for communication at the network layer (Layer 3). It allows you to confirm whether the client has successfully obtained a valid IP address, which indicates proper DHCP functionality. Troubleshooting can also involve verifying that the client's IP address is in the correct subnet, as incorrect IP configurations can prevent the client from accessing resources.

Troubleshooting Tools and Logs: With both the IP and MAC addresses, you can use diagnostic tools like ping, traceroute, nslookup, and Wi-Fi controller logs to trace the client's path through the network, identify where packets may be dropped, and determine if there is an issue with routing, DNS resolution, or firewall settings.

Analyzing Access Point Logs: In many enterprise Wi-Fi environments, controllers and access points log connectivity events based on clients' MAC addresses. Having the MAC address allows you to search logs for specific connection issues like authentication failures, association issues, and disconnections, helping you pinpoint root causes and determine whether the issue is specific to the client or network-wide.

Avoiding IP Conflicts: IP address conflicts can cause clients to lose connectivity. Checking the client's IP helps ensure there are no duplicate IP addresses, which can occur if multiple clients are assigned the same IP.

Security and Access Control: If access restrictions are in place (e.g., MAC filtering or IP-based ACLs), knowing the client's IP and MAC addresses helps you verify if these security mechanisms are affecting the client's connectivity.

Can they see the SSID? Knowing whether a Wi-Fi client can see the SSID (Service Set Identifier) is crucial when troubleshooting Wi-Fi connectivity issues because it provides insight into potential problems at the physical (Layer 1) and data link (Layer 2) layers of the network.

Here's why it's important:

Signal Range and Coverage Verification: If the client cannot see the SSID, it may be outside the range of the Wi-Fi signal, meaning it's too far from the access point (AP) or in an area with poor coverage. Knowing whether the client can see the SSID helps determine if signal strength or coverage could be the issue, which might require moving the client closer to the AP or addressing coverage gaps.
Channel and Frequency Band Compatibility: Different APs and clients support various channels and frequency bands (2.4 GHz, 5 GHz, and possibly 6 GHz). If the client doesn't see the SSID, it could be due to incompatibility with the frequency band or channel being used. For instance, if the client device only supports 2.4 GHz, it won't see an SSID broadcasting exclusively on 5 GHz.
SSID Broadcasting Settings: Some networks hide the SSID (disable SSID broadcasting) for security reasons. If the SSID is hidden, clients won't see it in the list of available networks and will need to manually enter the SSID to connect. Knowing whether the SSID is visible can help confirm whether a hidden SSID is causing confusion for users trying to connect.
Interference Issues: If the SSID is intermittently visible, it might indicate interference issues, which can affect the Wi-Fi signal quality. Nearby devices, such as microwaves, Bluetooth devices, or other Wi-Fi networks on the same channel, can cause interference, leading to inconsistent visibility of the SSID. This can help identify if interference is a possible factor impacting connectivity.
AP Configuration and Status: If no clients can see the SSID, it may indicate that the AP is down, misconfigured, or not broadcasting the SSID as expected. This information can help narrow down whether the issue is specific to a single client or if it affects multiple devices, potentially pointing to an AP or network configuration problem.
Security Settings and Access Control: In enterprise networks, some SSIDs are broadcast selectively based on user roles or device types. Knowing if a client can see the SSID helps determine whether access control policies might be preventing the SSID from being shown to specific clients.
Perform a DNS test: Having the client perform a DNS test from the command line is an important troubleshooting step in diagnosing Wi-Fi client issues because it helps identify problems related to domain name resolution.

Here are the key reasons why a DNS test is useful:

Verifying DNS Configuration: A DNS test helps confirm that the client has the correct DNS server settings. If the client cannot resolve domain names, it may be using incorrect or unreachable DNS servers, which can often happen if the DHCP configuration is incorrect or if there are network misconfigurations.
Confirming Internet Connectivity Beyond Basic Network Connection: Successfully connecting to Wi-Fi and obtaining an IP address does not guarantee full internet connectivity. DNS issues can cause a client to appear connected to Wi-Fi but unable to access websites or services by name. A DNS test (e.g., nslookup example.com or ping example.com) verifies whether the client can translate domain names to IP addresses, an essential step for normal internet browsing.
Isolating DNS-Specific Issues from General Connectivity Problems: If a client can successfully ping an IP address (like Google's 8.8.8.8) but cannot resolve domain names, it indicates that the problem is specific to DNS rather than general connectivity. This helps narrow down the issue, potentially pointing to a DNS server problem rather than a broader network issue.
Detecting Latency or Slow Response Issues: By testing DNS resolution directly, you can observe how quickly the DNS server responds. Slow DNS response times can cause delays in web page loading, application access, and overall connectivity performance. Running a DNS test can reveal if there are latency issues or timeouts when resolving domain names.
Identifying DNS Caching Issues: Sometimes, stale DNS cache entries cause resolution failures or lead to outdated IP addresses for certain domains. Running a DNS test helps verify if flushing the DNS cache (using dscacheutil -flushcache on macOS, for example) is necessary, especially if only certain sites are unreachable.
Testing with Alternative DNS Servers: Performing a DNS test allows you to easily switch to alternative DNS servers (e.g., Google's 8.8.8.8 or Cloudflare's 1.1.1.1) to rule out issues with the default DNS provider. If the test succeeds with an alternative DNS, it confirms that the issue lies with the client's primary DNS server rather than with the network itself.
Verifying Application and Service Access: Many applications and online services rely on DNS to locate servers. If DNS resolution fails, clients cannot access these applications. A DNS test helps identify if connectivity issues with specific services (e.g., email, cloud apps) are due to DNS problems.

What is their machine name?

Knowing the machine name (hostname) of the client's computer is important for Wi-Fi troubleshooting for several reasons:

Device Identification on the Network: The machine name helps identify the specific device on the network among multiple clients. Network administrators can easily locate the device in network logs, Wi-Fi controller interfaces, or DHCP server records, which can speed up the troubleshooting process.
Access Point and Controller Logs: Many enterprise-grade access points (APs) and Wi-Fi controllers log events by the device's hostname, MAC address, or IP address. Knowing the machine name allows the administrator to search logs for connection attempts, authentication events, or disconnections specific to that client, helping diagnose issues like frequent disconnects or access restrictions.
DHCP Server Tracking: The DHCP server often records the client's hostname alongside its IP address, allowing administrators to verify that the client received a valid IP address. This is essential for determining if the issue lies with IP address allocation, lease conflicts, or potential DHCP scope exhaustion.
Network Security and Access Control: Some networks enforce policies based on device names or restrict network access to known devices. Knowing the machine name can help verify if the device is allowed on the network and troubleshoot any access control or security policy issues that might prevent the client from connecting.
IP Conflict Resolution: When devices have conflicting IP addresses on the network, knowing the hostname can help identify the conflicting devices quickly. This is especially useful if DHCP servers, APs, or controllers report multiple devices trying to use the same IP.
Remote Troubleshooting and Support: If remote support tools are in use, knowing the device's hostname helps ensure the support team accesses the correct device for troubleshooting. In cases where remote commands are needed, the hostname allows for more targeted assistance, especially on larger networks.
Device Context and History: Knowing the machine name can provide context, such as recognizing a device that has had recurring issues. It can also help identify the specific OS, configuration, or role of the device on the network (e.g., whether it's a personal laptop, a work computer, or a shared device), allowing for more tailored troubleshooting steps.
Is the problem with a specific application or all applications?
Which application(s)?
Knowing whether a Wi-Fi client issue affects just one application, or all applications is critical in pinpointing the root cause because it helps differentiate between network-level issues and application-specific problems. Here's why this distinction is important:
Isolating Network vs. Application Issues: If only one application is experiencing issues, the problem is likely with that application, its configuration, or its server rather than with the Wi-Fi network itself. Conversely, if all applications are affected, it's more indicative of a network-related issue.
Diagnosing DNS or Connectivity Problems: Many applications rely on DNS to resolve domain names for accessing servers. If all applications are affected but specific IP-based commands like ping work, it might indicate a DNS issue. Knowing whether the problem is application-specific helps focus troubleshooting on DNS versus general network connectivity.
Application Requirements: Some applications require higher bandwidth, lower latency, or specific port access, which can lead to issues if the network doesn't meet these requirements. For example, video conferencing apps need stable, high-speed connections with low latency. If only the video conferencing app is experiencing issues, it could be due to network congestion, insufficient bandwidth, or firewall restrictions rather than a general Wi-Fi problem.
Firewall and Security Policies: Some networks have firewalls or security policies that block or limit access to specific applications or services. If only one application fails, it may be due to firewall restrictions, application-level filtering, or port-blocking policies that are affecting that particular app. Knowing this helps focus on inspecting firewall rules and security policies.
Server or Application-Specific Outages: Sometimes, an application's servers may be experiencing issues or outages. If only one application is affected, this could indicate that the problem is on the application server side rather than the Wi-Fi network. This distinction can prevent unnecessary troubleshooting of the network itself.
Performance Optimization: If a single application is slow or unreliable, the client may be experiencing issues with that app's configuration, such as outdated software, incompatible settings, or caching problems. In this case, troubleshooting should focus on optimizing or reinstalling the application rather than the network.
Guiding Further Tests and Diagnostics: Knowing whether one or all applications are affected helps tailor diagnostic tests. For instance, if all applications are affected, running connectivity tests (e.g., ping, traceroute) is more relevant. If only one application is affected, you might focus on testing that app's connectivity requirements, checking for updates, or examining server-side settings.

Is the problem with just their machine? Are more than 5 people experiencing the same issue?

Knowing if a Wi-Fi issue is limited to a single machine or affects multiple users is essential in troubleshooting because it helps determine whether the root cause is specific to the client's device or indicative of a broader network problem.

Here's why this distinction is crucial:

Identifying Network-Wide Issues: If more than five users are experiencing similar Wi-Fi problems, it points to a potential issue with the network infrastructure, such as an access point (AP) malfunction, DHCP server issues, or network-wide configuration errors. This information shifts focus from the client device to network equipment, network settings, or interference issues affecting multiple users.
Determining Scope and Impact: Widespread connectivity problems could indicate a significant outage or misconfiguration that needs immediate attention. Understanding that the issue affects multiple users helps prioritize troubleshooting efforts and resources, potentially involving network administrators or IT support teams for a faster resolution.
Isolating Device-Specific Issues: If only one client is experiencing the problem, it's likely due to something specific to that device, such as incorrect network settings, outdated Wi-Fi drivers, software conflicts, or hardware issues. This knowledge allows troubleshooting efforts to focus on that client's machine, saving time and preventing unnecessary changes to network settings.
Differentiating AP or Coverage Issues: If multiple people near a specific area are having problems, it might indicate an issue with a specific AP or signal coverage in that zone. Knowing that others nearby are affected helps identify AP placement issues, power or channel misconfigurations, or interference in that area, while a single-device issue would typically rule these out.
Narrowing Down Configuration and Authentication Problems: When a large number of users experience similar problems, it could stem from network configuration issues, such as DHCP or DNS server issues, authentication server problems, or security certificate issues. For example, if a change was made to the network's authentication settings and multiple users are affected, it's likely an issue with how the network is configured rather than with individual devices.
Guiding Communication and User Expectations: When an issue is widespread, notifying users and setting expectations can reduce confusion and frustration. This also helps reduce the influx of individual support tickets by letting users know it's a known issue that is being addressed.

If somebody says the wireless network is slow, you're going to need the basics. You need to know if they're on Mac/Windows, their IP address or MAC address.

Then ask questions.What do they mean when it gets slow?Was it just one application?Is it multiple applications?Is it just them?Is it everybody in their department?

Places to look in the CiscoMeraki dashboard for data on a client device:

Network-wide>Clients>Search for clients> [client IP, MAC or 802.1x username]
Overview>Usage for the last 30 days
Policy>Show Details
Performance

If they say that the Wi-Fi isn't working, ask them to elaborate on what they mean by not working. You will need their IP address or MAC address and more information on what was happening when the Wi-Fi stopped "working".

Places you can look in the CiscoMeraki dashboard to get data on client connections over time:

Overview>Current Client Connection and Timeline>History

If someone says the Wi-Fi is dropping, this speaks to people being in video conference calls and experiencing the video conference stream freezing up or the video dropping out. The audio may be getting jittery.

What do they mean when they say the Wi-Fi is dropping?Can you elaborate on what you were doing when the Wi-Fi began dropping?Same questions about their IP address or MAC address.

Overview>Current Client Connection

If someone says the Wi-Fi is "down": Navigate to Timeline>History Clients>Overview>Current Client Connection> Click on current AP and choose "View more details" Wireless>Access Points. This will show a list of all access points for this location. The status icon next to each access point will indicate if the AP is offline, alerting or online. Click on the AP name to find out why it is alerting.

In this instance, the AP is in "unplanned low-power mode" which may cause certain features of the access point to be non-operational. Each model of access point will have different features which cannot be enabled when the access point is in low-power mode. Consult the specific model of access points' whitepaper to find out what is disabled when the access point is in low-power mode.

Accessing the Wireless Network Overview: Log into the Meraki Dashboard at https://dashboard.meraki.com. Navigate to Wireless > Monitor > Overview. Review the Connection, Performance, Network Service Health section for insights on network stability and key metrics. Note any anomalies, such as significant fluctuations in the Usage Overview statistic, which may indicate network instability.

Reviewing Client Connectivity Logs: Go to Network Wide > Clients to view a list of devices connected to the wireless network. Review specific performance for each client by selecting the client's name, then go to Overview Connections, Performance, Roaming, Timeline or Stored Captures in the client's profile.

Checking for Dropped Connections and Authentication Failures: Navigate to Network-wide > Event Log. Choose Event log for access points to view the 802.1x event log history Filter according to 802.11 association/disassociation, 802.1x authentication/deauthentication. Investigate repeated patterns or specific timeframes with a higher frequency of these events.

Validating Access Point Status and Connectivity: Go to Wireless > Access Points to view a list of all APs in the network. Check the Connection Health for the overall APs, which reflects failed clients, time to connect and roaming for wireless clients. Identify APs with poor signal strength or connectivity errors; drill down to Performance Health to view Latency, Packet loss and Signal quality (SNR) values. Confirm all APs show an Online status and an acceptable channel utilization and packet loss under Wireless > RF Spectrum.

RF and Spectrum Analysis: In Wireless > RF Spectrum, validate channel utilization and packet levels.

Click into each AP to view Interfering Aps (same channel), Interfering Aps (some overlap), Non-Interfering APs and Total In-Network Neighbors.
Utilize the Live Tools feature select the access point of interest from the Monitor > Access points page to run real-time ping tests or packet capture on specific APs for deeper diagnostics if performance issues are detected.

Relevant Documentation

Regularly Audit Administrator Accounts:

Periodically review all active administrator accounts and their assigned permissions.

Best Practice: Perform audits at least quarterly to identify unused or misconfigured accounts and adjust permissions as needed.

Use Role-Based Access Control (RBAC): Define access controls based on administrator responsibilities rather than personalizing permissions.

Best Practice: Update roles and permissions when an administrator's responsibilities change rather than creating new permissions for specific users.

Disable Inactive Accounts: Immediately remove access for any users who no longer require it, such as terminated employees or personnel who have shifted roles.

Best Practice: Implement automated processes for disabling accounts when staff departs to prevent unauthorized access.

Document Administrator Policies: Create a policy document for onboarding new administrators, detailing access requirements, authentication methods, and MFA setup.

Best Practice: Require administrators to review and acknowledge policies during onboarding.

Network Monitoring

Monitoring the wireless network in real time helps ensure continuous performance and identify potential issues.

Wireless Health: Regularly monitor the Wireless Health section in Wireless > Monitor > Health for a high-level overview of client connectivity and potential issues with association, authentication, and DHCP.

Best Practice: Set up alert notifications for issues such as access point downtime, high association failures, and excessive packet loss. Review these alerts daily or weekly.

RF Spectrum Analysis: Use RF Spectrum tools under Wireless > Monitor > RF Spectrum to monitor interference and channel utilization, which can directly impact network performance.

Best Practice: Adjust channels for APs in areas with high interference, or enable Auto RF if it's not yet configured to automatically adjust to optimal channels.

Uplink Monitoring: Go to Wireless > Monitor > Access Points to view each access point's status and connectivity.

Best Practice: Regularly check for high latency, packet loss, and failed uplink connections on APs, especially in critical areas with high traffic.

Relevant Documentation

Client Management and Troubleshooting

Proactively managing client connectivity and handling client issues promptly are essential for a seamless user experience.

Client List: Access Network-Wide > Monitor > Clients to see a comprehensive view of all devices, including their connection status, bandwidth usage, and SSID.

Best Practice: Review clients frequently to identify high-bandwidth users or devices that may be causing network congestion.

Client Event Logs: Select a client from the Clients list to review specific Event Logs for each connection attempt, including association and authentication events.

Best Practice: For clients facing persistent connectivity issues, use the Event Log to diagnose problems such as signal strength, association failures, or authentication errors.

Block/Unblock Clients: From the Clients page, block clients that abuse bandwidth or do not adhere to network policies.

Best Practice: Set up policies to automatically restrict bandwidth for specific client types or SSIDs as needed.

Logging and Auditing

Wireless Optimization

Fine-tuning wireless settings optimizes performance and enhances user experience across the network.

Auto RF Optimization: Enable Auto RF in Wireless > Radio Settings for automated channel and power settings based on real-time RF conditions.

Best Practice: Enable Auto RF on networks with numerous APs to reduce co-channel interference and adjust AP power levels automatically.

Channel Width and Band Selection: Configure appropriate channel widths (e.g., 20 MHz for 2.4 GHz and 40 or 80 MHz for 5 GHz) to balance performance with range and interference.

Best Practice: Use 5 GHz whenever possible to avoid congestion in the 2.4 GHz range. Limit SSID Broadcast on 2.4 GHz to reduce interference on large networks.

Band Steering: Enable Band Steering in Wireless > Configure > Radio Settings > RF profiles to encourage clients to use 5 GHz over 2.4 GHz for better performance.

Best Practice: Configure Band Steering on SSIDs where most devices support 5 GHz, ensuring that newer devices utilize the less congested frequency band.

Relevant Documentation

Reporting and Insights

Use the Meraki Dashboard's reporting tools to gather insights into network usage, identify trends, and create reports for further analysis.

Usage Reports: Go to Network-wide > Traffic Analytics to view data on bandwidth consumption by application, user, and SSID.

Best Practice: Generate weekly or monthly usage reports to understand trends, such as high-bandwidth applications, to make informed adjustments on QoS and bandwidth policies.

Access Point Summary Reports: In Organization > Summary > Networks > Usage and clients > SSID, generate reports on SSID applications, ports, HTTP content, IP version, and Trusted Traffic statistics.

Best Practice: Review AP summaries monthly to evaluate if additional APs are needed in high-density areas or if any APs require relocation or reconfiguration.

Scheduled Reports: Set up automated reporting via Syslog, API/Webhooks, or SNMP alerting to send IT staff regular updates.

Best Practice: Schedule reports weekly to track the network's health, usage trends, and troubleshoot recurring issues without logging in daily.

Relevant Documentation

Logging and Auditing

Regular auditing of administrative actions is essential to track changes and identify potential security incidents:

Event Logs: In Organization > Change Log, you can view recent administrative actions, including login attempts, configuration changes, and permission updates.

Best Practice: Review logs weekly for unusual activity or unauthorized configuration changes. Look specifically for failed login attempts or changes made outside business hours.

Automated Alerts: Set up alerts for critical events such as multiple failed login attempts or unauthorized changes to network settings. Organization > Monitor > Login Attempts

Best Practice: Ensure that alerts are sent to a shared IT mailbox or ticketing system so that they are not missed.

If you haven't already done so, create an account on WWT.com to access all of WWT's knowledge and contact the WWT Global Engineering Team with your questions.

Cisco Meraki: A Day In The Life

Here's why the client's OS is important:

Here's why it's important:

Here are the key reasons why a DNS test is useful:

What is their machine name?

Knowing the machine name (hostname) of the client's computer is important for Wi-Fi troubleshooting for several reasons:

Here's why this distinction is crucial:

Technologies