Recently global issue happened where license is missing from License manager. When i try reapplied the license, it keeps kicking and deleting.

Issue Description

• Users experience access denied

  

• The Citrix License Manager interface does not show the expected licenses.

• Applications relying on Citrix licenses become inaccessible.

• The issue appears to be widespread among those using Citrix License manager version 11.17.2.0 build 51000.

Solution

1.1 Back Up License Files

• Navigate to:

    C:\Program Files (x86)\Citrix\Licensing\MyFiles
  

• Copy all .lic files to a safe location.

1.2 Export Citrix License Server Settings

• Open Registry Editor (regedit).

• Go to:

    HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\Licensing
  

• Right-click on the Licensing folder and choose Export.

• Save the file as Citrix_License_Backup.reg.

1.3 Backup the License Server Database

• Navigate to:

    C:\Program Files (x86)\Citrix\Licensing\LS
  

• Copy the db folder to a backup location.

Step 2: Uninstall Citrix Licensing Manager

To successfully downgrade, you must remove the newer version first.

2.1 Stop Licensing Services

• Open Services.msc and stop:

  • Citrix Licensing

  • Citrix License Management Service

2.2 Uninstall Citrix Licensing Manager

• Go to Control Panel > Programs and Features.

• Find Citrix Licensing Manager.

• Click Uninstall and follow the prompts.

2.3 Delete Residual Files

After uninstalling, delete leftover files:

• Navigate to:

    C:\Program Files (x86)\Citrix\Licensing
  

• Delete any remaining files.

• Remove Citrix Licensing registry entries:

    HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\Licensing
  

• Restart your server.

Step 3: Download and Install Version 49000 or Lower

3.1 Download the Older Version

• Visit Citrix Download Center.

• Search for Citrix Licensing Manager 11.16.6.0 build 49000 or your preferred older version.

• 

• Download the .exe installer.

3.2 Install the Older Version

• Run the installer with Administrator privileges.

• Follow the setup wizard and complete the installation.

• Restart your server after installation.

Step 4: Restore Backups and Reconfigure

4.1 Restore License Files

• Copy the .lic files from your backup to:

    C:\Program Files (x86)\Citrix\Licensing\MyFiles
  

4.2 Restore Registry Settings

• Double-click the Citrix_License_Backup.reg file.

• Confirm the merge.

4.3 Restore Database Files

• Copy the backed-up db folder to:

    C:\Program Files (x86)\Citrix\Licensing\LS
  

Step 5: Restart Services and Verify

5.1 Restart Licensing Services

• Open Services.msc.

• Start:

  • Citrix Licensing

  • Citrix License Management Service

5.2 Verify License Availability

• Open Citrix Licensing Manager via:

    https://<your-license-server>:8083
  

• Ensure licenses are recognized and assigned correctly.

5.3 Test License Assignment

• Run:

    lmstat -a -c @<LicenseServerName>
  

• Confirm licenses are displayed.

There is 2 options for renewal.

Beta-License Renewal

Get your license when you have Citrix account. Go to Previews/Betas - License Retrieaval

Update the license in Citrix Studio. Go to Licensing > Allocate License and enter the serial number

Reinstall again Citrix studio.

Uninstall expired Delivery Controller. and Install again ISO.

Mount ISO and install again new Citrix Studio

Normally i will choose Virtual Apps and Desktop

Choose default

By right, it will not show expired anymore once reinstall finished

SSL certificates are crucial for securing communications between clients and servers by encrypting data and ensuring authenticity. SSL certificates have expiration dates, typically every 1 to 2 years, to maintain security standards and compliance.

Failing to renew an SSL certificate can lead to service disruptions, security warnings, and potential loss of trust from users.

Generate a New CSR (If Required)

1. Log in to Citrix ADC GUI.

2. Navigate to Traffic Management > SSL > SSL Files.

3. Click on Create RSA Key:

   • Key Filename: mydomain.key

   • Key Size: 2048 (recommended)

   • PEM Encoding: Enable

4. Click Create.

5. Generate a CSR:

   • Go to SSL > SSL Files > Create CSR.

   • Select the RSA key file you just created.

   • Fill in the required details (Common Name, Organization, Country, etc.).

   • Click Create.

   • Download the CSR file and submit it to the CA.

Locate the Expiring Certificate

1. Login to Citrix ADC GUI.

2. Go to:

   • Traffic Management > SSL > Server Certificates

3. Locate the expiring certificate.

4. 

   Hover over the certificate, click the "3 dots" menu, and select "Show Bindings".

5. Record Certificate Bindings: Take a screenshot or note down all bound services

6. 

   Can check validity of public certificates by renaming the certificate to .cer. Example: 25-26-PublicCert.pem to .cer

   

   Upload Certificate to NetScaler

   1. Navigate to:

      • Traffic Management > SSL > Certificates > All Certificates

   2. Click Install.

   3. Enter the details

      • Certificate-Key Pair Name: [any suitable name]

      • Certificate File Name: [certificate either in crt, PEM etc]

      • Key File Name: [private key file used to generate the certificate.]

      • Certificate format: PEM

        Citrix ADC primarily uses PEM format, which contains:

        • Certificate (.crt or .pem)

        • Private Key (.key)

        • Intermediate CA Certificates (if applicable)

Bind the Certificate to Virtual Servers

1. Go to:

   • Traffic Management > Citrix Gateway > Virtual Servers

2. Identify the virtual server using the expiring certificate.

3. 

   Hover over it, click “3 dots”, select "Edit".

4. Click "1 Server Certificate" to open Certificate Bindings.

   

5. Unbind the old certificate:

   • Hover over the old certificate, click "3 dots" → Select Unbind.

6. Bind the new certificate: Click Add Binding

7. Select the newly installed certificate.

8. Click OK > Save Changes

Update SSL Gateway Virtual Servers

1. Navigate to:

   • Configuration > Citrix Gateway > Citrix Gateway Virtual Servers

2. Locate SSL Gateway using the old certificate.

3. Click Edit → Select Certificate Details.

4. Bind the new one

Validate and Verify Certificate Renewal

Run the following command: show ssl certKey [Certificate key-pair name] Ensure the expiry date is correct.

Test SSL Connectivity

1. Open a web browser.

2. Navigate to the website, click the lock icon, and view certificate details.

openssl s_client -connect yourdomain.com:443 -servername yourdomain.com

Sync to Secondary ADC [force sync]

show ha node

sync ha files

force ha sync

show ha node

Upgrading Citrix NetScaler is essential for:

• Security Enhancements: Patches and fixes for known vulnerabilities.

• Performance Improvements: Optimized traffic management and processing.

• New Features: Access to advanced networking and security functionalities.

• Compatibility: Ensures seamless integration with newer Citrix infrastructure and third-party applications.

• Bug Fixes: Resolves known issues and improves system stability.

While Citrix Netscaler provide GUI based upgrade process, it is not recommended method due to several factors like stability and reliability. GUI is known to occasionally free, time out or fail. It has higher risk of corrupting the system due to not properly clean up old files.

Based on experience, success rate upgrade using GUI is around 70%++ only.

Upgrade NetScaler Implementation Plan

Pre-Upgrade Planning:

• Before upgrading, create a full virtual machine (VM) snapshot if NetScaler is running on a virtualized environment. This ensures a quick recovery in case of upgrade failure. Ensure sufficient storage space for the snapshot.

• Log in to vCenter.

• Locate your NetScaler virtual machine.

• Create a snapshot for the NetScaler

Create Full Backup via Web Interface

• Log in to NetScaler using your web browser.

• Navigate to Configuration > System > Backup and Restore.

• For naming format: File name [DD-MM-YY], select full in level option

• Create a full backup and download it to your local machine

Filename: [date]

level: Full

Comment: Any reference like CHG number

Backup ns.conf using WinSCP

Open WinSCP.

• Enter the IP address of your NetScaler, username, and password. Use SCP protocol

• Click "Login".

• Navigate to the /flash/nsconfig directory.

• Backup the ns.conf file by copying it to your local machine

• This backup allows quick restoration of the primary configuration file in case of errors

Upgrade NetScaler

• In WinSCP, navigate to the /var/nsinstall/Builds directory

• Create a new folder named after the firmware version you are upgrading to. Normally need to download the firmware from https://www.citrix.com/downloads/

• Upload the firmware file to this newly created folder.

• Open putty and Enter the IP address of your NetScaler

• Enter the IP address of your NetScaler and connect

Access Shell:

• Once connected, type shell to access the shell prompt

Check Disk Space:

• Type df -h to view the available disk space.

• Ensure the /var partition has at least 4GB of free space.

Navigate to Firmware Directory:

• Type cd /var/nsinstall/ to navigate to the firmware directory.

• Type ls to see all the list that available

Extract Firmware:

• Type tar -xvzf to extract the firmware files.

It stands for:

x → Extract: This tells tar to extract files from an archive.

v → Verbose: Displays the files being extracted, providing progress output.

z → Gzip: Specifies that the archive is compressed using gzip (.tgz or .tar.gz format).

f → File: Indicates that the next argument is the filename of the archive.

• Type ls -tl once the extraction completed to see list is available

Install Firmware:

• Type installns to start the firmware installation process.tar

• Respond to any confirmation prompts as needed. Delete old signature files and kernel images if asked

Wait for Reboot:

• The NetScaler will automatically reboot once the firmware installation is complete.

• Can monitor the process at the vCenter

Verify Firmware Version:

• After the reboot, log in to the NetScaler via the web interface.

• Make sure it does not change to freemium

• Check the firmware version by clicking on the nsroot account in the top right corner.

• Can also try validation by launching the storefront

Ensure 4GB of /var disk space is available. By checking on the configuration > System Upgrade > Check Disk Space

Initially, I used Flask to build a simple password generator. However, I found it a bit complex since it required creating separate HTML files for the website.

Then I discovered Streamlit, which made the process much easier. With Streamlit, you can focus on your Python code and let it handle the web interface for you.

In this post, I'll first walk you through the code, and then I'll show you how to create a free web app using it.

At first, will show the algorithm for password generator. Its pretty straight forward.

def generate_random_password(length=12):
    # string.ascii_letters is for lowercase and uppercase letters
    # string.digits is for numbers
    # string.punctuation is special characters
    characters = string.ascii_letters + string.digits + string.punctuation
    password = ''.join(random.choice(characters) for index in range(length))
return password

For the function, It starts with an empty string ('') and adds characters to it using the join() function.

for index in range(length)) used to pick up random characters till the length is achieved.

That’s the basic idea. Now, let’s move on to the fun part: making a web app.

Website link

import random
import string
import time
import streamlit as st

funny_quotes = [
    "This password is so strong, it does push-ups!",
    "Even your mom can't guess this one!",
    "MCMC is watching",
    "Guard this password like it's your last slice of pizza!",
    "Better write this one down—it's practically a masterpiece!",
    "Your password is more complicated than assembling IKEA furniture.",
    "Your password is now as secretive as a magician’s tricks.",
    "This password is so good, even the MCMC would be impressed.",
    "This password is so secure, even Sherlock Holmes couldn’t deduce it.",
    "This password is more complicated than your love life.",
    "This password is stronger than your morning coffee.",
    "This password is more mysterious than the Bermuda Triangle.",
    "This password is harder to guess than the plot twist in a Momento.",
    "This password is harder to figure out than the meaning of life.",
    "This password is more confusing than your partner.",
    "This password is more secure than your emotional walls.",
]

def generate_random_password(length=12):
    characters = string.ascii_letters + string.digits + string.punctuation
    password = ''.join(random.choice(characters) for _ in range(length))
    return password

st.title("Random Password Generator")

length = st.slider("Select password length", min_value=8, max_value=30, value=12)

if st.button("Generate Password"):
    password = generate_random_password(length)
    st.write("Generated password:")

    # Display password in a text input for easy copying
    st.header(password)

    # Temporary pop-up message for success
    success_placeholder = st.empty()
    success_placeholder.success("Password generated successfully.")
    st.balloons()

    # Wait for a few seconds
    time.sleep(3)

    st.write(random.choice(funny_quotes))

    # Clear the success message
    success_placeholder.empty()

# Simple tagline below the page
st.write("\n" * 40)  # Adding some space before the footer
st.write("ahmadafif.com", align="center", font_size=17)

For the library we will import few:

import random
import string
import time
import streamlit as st

random and string: to generate random characters and string for password

time: for brief delay before display quote

streamlit: library that create interactive web app

you can install the library by using this command:

pip install streamlit

At first, I hard-coded the funny quotes like below

funny_quotes = [
    "This password is so strong, it does push-ups!",
    ...
]

password generator is the same as explained earlier

for web interface:

st.title("Random Password Generator")
length = st.slider("Select password length", min_value=8, max_value=30, value=12)

Title: The app is titled "Random Password Generator".

Password Length Slider: You can choose the length of the password using a slider, with options ranging from 8 to 30 characters.

Generate Button and Display:

if st.button("Generate Password"):
    password = generate_random_password(length)
    st.write("Generated password:")
    st.header(password)

Button: When you click the "Generate Password" button, the app creates a new password.

Display: The generated password is shown on the screen, ready for you to copy.

Fun Extras:

success_placeholder = st.empty()
success_placeholder.success("Password generated successfully.")
st.balloons()
time.sleep(3)
st.write(random.choice(funny_quotes))
success_placeholder.empty()

Success Message and Balloons: After generating the password, a success message appears, and fun balloons fill the screen.

Funny Quote: After a brief delay, a random funny quote from the list is displayed, adding a bit of humor to the experience.

How to upload to web apps

1. Run in local

   Before deploying, you should test your app locally to ensure everything works. In the terminal or command prompt, navigate to the directory where your [filename].py file is located and run:

streamlit run [filename].py

This command will open a new tab in your web browser where you can see your app in action.

2. Create a GitHub Repository

   Go to your GitHub profile and click on "New" to create a new repository.

   Name your repository (e.g., PasswordGeneratorStreamlite) and choose to make it public.

   Upload your code.

   It will be something like this:

   

3. Deploy app to Streamlit

   visit website and login using Github account (prefferable)

   Click on "New app" and select the GitHub repository you just created.

   

   Choose the branch (usually main or master) and the Python file (e.g BasePassGen.py).

   Click "Deploy".

   Congratulations! Your app is ready.

1. Go to Command Prompt (CMD)
2. Type "shutdown -i"
3. Add servers, computer list that you want to shutdown or restart
4. Put comment why restart
5. Enter ok.

$list = Get-Location
$serverListPath = "$list\citrixProfileManagementList.txt" #create one later file


$logFilePath = "$list\serviceLog.log" #will create later


$logentries = @()
$sessions = @()


$servers = Get-Content -Path $serverListPath


foreach($server in $servers){
    try {
        $session = New-PSSession -ComputerName $server -ErrorAction Stop
        $sessions += $session
    }
    catch{
        $logentry = "Server: $server - Error to create session - $_"
        $logentry += $logentry
        Write-Host $logentry
    }

}

foreach($session in $sessions){
    $serverName = $session.ComputerName
    try{
        $checkService = Invoke-Command -Session $session -ScriptBlock { Get-Service -Name "ctxprofile" -ErrorAction stop}  

        if($checkService.status -eq "running"){
            $logentry = "server $serverName  - service ctxprofile is running"
        }
        else {$logentry = "server $serverName  - service ctxprofile is not running" }

    }
    catch{
        $logentry = "Service ctxProfile not found on server: $server -error: $_"
    }

    $logEntries += $logentry
    Write-Host $logentry
}

#need to close all Pssessions
<#$sessions | ForEach-Object {
    Remove-PSSession -Session $_
} #>

#or can use like this
foreach($session in $sessions){
    Remove-PSSession -Session $session
}


$logentries | Out-File -FilePath $logFilePath -Force
$logentries | Out-GridView -Title "Service Check Results"

For checking the service is running only and outcome in Out-Grid View (OGV), can use this powershell script

$list = Get-Location
$serverListPath = "$list\citrixProfileManagementList.txt" #create one later file


$logFilePath = "$list\serviceLog.log" #will create later


$logentries = @()
$sessions = @()


$servers = Get-Content -Path $serverListPath


foreach($server in $servers){
    try {
        $session = New-PSSession -ComputerName $server -ErrorAction Stop
        $sessions += $session
    }
    catch{
        $logentry = "Server: $server - Error to create session - $_"
        $logentry += $logentry
        Write-Host $logentry
    }

}

foreach($session in $sessions){
    $serverName = $session.ComputerName
    try{
        $checkService = Invoke-Command -Session $session -ScriptBlock { Get-Service -Name "ctxprofile" -ErrorAction stop}  

        if($checkService.status -eq "running"){
            $logentry = "server $serverName  - service ctxprofile is running"
        }
        else {$logentry = "server $serverName  - service ctxprofile is not running" }

    }
    catch{
        $logentry = "Service ctxProfile not found on server: $server -error: $_"
    }

    $logEntries += $logentry
    Write-Host $logentry
}

#need to close all Pssessions
<#$sessions | ForEach-Object {
    Remove-PSSession -Session $_
} #>

#or can use like this
foreach($session in $sessions){
    Remove-PSSession -Session $session
}


$logentries | Out-File -FilePath $logFilePath -Force
$logentries | Out-GridView -Title "Service Check Results"

Why Update NetScaler?

Updating NetScaler is crucial for several reasons:

• Security: when the IT security guy chasing you for vulnerabilities

• Performance: Benefit from performance improvements in newer versions.

• Features: Access new features and enhancements.

1. Install WinSCP:

• Ensure you have WinSCP installed on your machine. You can download it from WinSCP official website.

2. Login to NetScaler using WinSCP:

• Open WinSCP.

• Enter the IP address of your NetScaler, username, and password.

• Click "Login".

  3. 

     Backup Configuration File:

• Navigate to the /nsconfig directory.

• Backup the ns.conf file by copying it to your local machine.

  

4. Create Full Backup via Web Interface:

• Log in to NetScaler using your web browser.

• Navigate to Configuration > System > Backup and Restore.

• For naming format: File name [DD-MM-YY], select full in level option

• Create a full backup and download it to your local machine.
  Create Snapshot via vCenter:

• Log in to vCenter.

• Locate your NetScaler virtual machine.

• Create a snapshot for the NetScaler.

6. Upload Firmware using WinSCP:

• In WinSCP, navigate to the /var/nsinstall directory.

  

• Create a new folder named after the firmware version you are upgrading to.

• Upload the firmware file to this newly created folder.

7. Run PuTTY:

• Open PuTTY.

  

• Enter the IP address of your NetScaler and connect.

8. Access Shell:

• Once connected, type shell to access the shell prompt.

9. Check Disk Space:

• Type df -h to view the available disk space.

• Ensure the /var partition has at least 4GB of free space.

10. Navigate to Firmware Directory:

• Type cd /var/nsinstall/<firmware_folder> to navigate to the firmware directory.

• Type ls to see all the list that available

11. Extract Firmware:

• Type tar -xvzf <firmware_name.tgz> to extract the firmware files.

• Type ls -tl once the extraction completed to see list is available

  

12. Install Firmware:

• Type installns to start the firmware installation process.

  

• Respond to any confirmation prompts as needed. Delete old signature files and kernel images if asked

13. Wait for Reboot:

• The NetScaler will automatically reboot once the firmware installation is complete.

• Can monitor the process at the vcenter

14. Verify Firmware Version:

• After the reboot, log in to the NetScaler via the web interface.

• Make sure it does not change to freemium

• Check the firmware version by clicking on the profile account in the top right corner.

• Can also try validation by launching the storefront or SSH to netscaler and type show version

  

Online websites for downloading YouTube videos are often full of ads and complicated to use. For educational purposes, here’s a simple, ad-free way to create your own YouTube video downloader using Python.

Result:

Lets get started!

Let me show you the essential script.

#pip install pytube (if module not installed yet)
from pytube import YouTube

#insert the link you want to download
yt = YouTube("https://www.youtube.com/watch?v=K9tOnIJyvRw")
#streams object able to use multiple method to call
yt.streams.get_highest_resolution().download()
print(f"Video downloaded successfully: '{yt.title}'")

That's all you need.

• yt.streams: This gets all available video streams for the YouTube video.

• get_highest_resolution(): This method selects the stream with the highest resolution.

• download(): This method downloads the selected video stream.

From the test, it seems that unable to call method directly but need to call method from object streams which is get_highest_resolution()

Full explanation:

You cannot call the download method directly on the YouTube object because the download method is a part of the Stream object, not the YouTube object. Here’s a more detailed explanation:

1. YouTube Object:

   • When you create a YouTube object, it represents a specific YouTube video. This object contains various attributes and methods to interact with the video.

2. Stream Object:

   • The YouTube object has a streams attribute, which is a list of Stream objects. Each Stream object represents a different format or quality option for downloading the video (e.g., 360p, 720p, audio-only, etc.).

3. download Method:

   • The download method is a method of the Stream class, not the YouTube class. Therefore, you need to first select a specific Stream object from the streams attribute of the YouTube object before you can call the download method.

We want to make it better by stating the path to save downloaded video and also URL for video.

#pip install pytube (if module not installed yet)
from pytube import YouTube

Importing Youtube class from pytube library which will be used to interacting with Youtube videos

# Create function to download a YouTube video
def download_video(video_url, output_path):
    try:
        video = YouTube(video_url)
        stream = video.streams.get_highest_resolution()
        stream.download(output_path=output_path)
        print(f"Video downloaded successfully: '{video.title}'")
    except Exception as e:
        print(f"Failed to download video. Error: {e}")

def download_video(video_url, output_path) is defining a function name as download video that takes two arguments which is video url and output path

• video_url: The URL of the YouTube video to be downloaded.

• output_path: The directory where the downloaded video will be saved.

video = YouTube(video_url) create 'Youtube' object for specified video URL that will called later as variable video

stream = video.streams.get_highest_resolution() Selects the highest resolution stream available for the video.

stream.download(output_path=output_path) Downloads the selected stream to the specified directory.

print(f"Video downloaded successfully: '{video.title}'") output if the video downloaded successfully

except Exception as e: print(f"Failed to download video. Error: {e}") when the download not able to process, this output will trigger

video_url = input('Enter URL of YouTube video: ') 
output_path = input('Enter the directory to save the video: ')

input means that user will need to fill the URL and also the path which will be stored to the variable video_url and output_path

download_video(video_url, output_path)

This last script will call download_video function with the argument that has been given by user.

FULL CODE

# pip install pytube (if module not installed yet)
from pytube import YouTube

# Function to download a YouTube video
def download_video(video_url, output_path):
    try:
        video = YouTube(video_url)
        stream = video.streams.get_highest_resolution()
        stream.download(output_path=output_path)
        print(f"Video downloaded successfully: '{video.title}'")
    except Exception as e:
        print(f"Failed to download video. Error: {e}")

# Request URL of YouTube Video
video_url = input('Enter URL of YouTube video: ')
output_path = input('Enter the directory to save the video: ')

# Download the video
download_video(video_url, output_path)

Let me know if you have any comment.

Managing multiple servers often involves repetitive tasks, such as installing specific software. Manual installation on each server can be time-consuming and prone to errors. I will demonstrate how to use PowerShell to automate the installation.

My sample application is Notepad++ which will be installed to several servers in my lab environment.

You can download the Notepad++ installer from the official website.

Solution:

Will use PowerShell script with few functions. In addition, will create log file for reference and troubleshoot purpose.

Result:

Using the PowerShell script, Notepad++ will be installed on multiple servers, and a log file will be created. This eliminates the need for manual installation on each server. Full code at the end of article.

Lets get started!

Pre-requisite

1. Ensure each server has access to the network location where the Notepad++ installer is stored.

2. Ensure PowerShell Remoting is enabled on all servers.

3. Ensure the user running the script has administrative privileges on all target servers.

In production environment, you need to make sure network and firewall settings allow for PowerShell Remoting and access to the network share.

$installerLocalPath = "C:\CopyFiles\npp.8.6.7.Installer.x64.exe"
$installerRemotePath = "C:\Program Files\npp.8.6.7.Installer.x64.exe"
$serverList = "C:\Copyfiles\server.txt"
$installerArguments = "/S"  # Silent install argument for Notepad++
$servers = Get-Content -Path $serverList
$logFilePath = "C:\CopyFiles\install_log.txt"

$installerLocalPath the variable for Notepad++ installer as source file

$installerRemotePath Destination where the installer will be copied

$serverList is text file that contain list of server. For test purpose I just use two servers in Lab Environment as below. Can use IP address but of course not convenient

$installerArguments is set to /S which the installer will run in silent mode without user interaction. There are more other option like:

/D=<path> to specify installation directory

/noDesktopShortcut will not create desktop shortcut

/noUpdater prevent automatic updates especially in controlled environment.

Best practice is specify installation directory and silent installation /S to avoid manual intervention

$servers = Get-Content -Path $serverList Reads the content for specified file in $serverList variable. Each line inside the file represent different server

Start-Transcript -Path $logFilePath -Append

Start a transcript which will log all output from the script. It will append to the file if it already exist.

foreach ($server in $servers) {

    try {
        $session = New-PSSession -ComputerName $server
        Copy-Item -Path $sourcepath -Destination $destinationPath -ToSession $session -Force
        Write-Host "Copy to $server successful"

foreach ($server in $servers) will create a loop based on server.

I'm using try to handle any error during execution of the command will pass information.

$session = New-PSSession -ComputerName $server will create PowerShell session to server specified which is variable $server

Copy-Item -Path $sourcepath -Destination $destinationPath -ToSession $session -Force :It will copy installer from $sourcePath and paste it to $destinationPath.

-ToSession $session is needed to specify to which remote session it will be copied. If not, it will copy the file locally and not into the remote servers

-force will overwrite if there is existing file

        # Command to install the software
        $installCommand = {
            param($installerPath, $installerArguments)
            Start-Process -FilePath $installerPath -ArgumentList $installerArguments -Wait}

param($installerPath, $installerArguments) there is two parameters. The installer path and also installer arguments. As we declare before installer path and $installerArguments we already set as /s which will be silent installation.

Start-Process -FilePath $installerPath -ArgumentList $installerArguments -Wait

Start-Process will start new process

-Wait will ask PowerShell to wait for the process to complete before proceeding to the next command. This ensures that the installation finishes before moving on.

Later, it will be use Invoke-command to run all parameters

Invoke-Command -Session $session -ScriptBlock $installCommand -ArgumentList $softwareInstallerPath, $installerArguments
        Write-Host "Software installation on $server successful"

Invoke-Command will run a command or script block on remote computers

It passes the installer path and arguments to the script block.

If success, output "Software installation on $server successful"

catch { Write-Host "Error during operation on {$server}: $_"}

Any errors that occur during the try block and outputs an error message to the console, including the server name and the error details

} finally { Remove-PSSession -Session $session Write-Host "Session ended for $server" }

Ensures that the PowerShell session to the server is removed. This is best practice to make sure after work completed, remove remote session.

Stop-Transcript

Stop the log. and it will generate the txt file.

FULL SCRIPT

$installerLocalPath = "C:\CopyFiles\npp.8.6.7.Installer.x64.exe"
$installerRemotePath = "C:\Program Files\npp.8.6.7.Installer.x64.exe"
$serverList = "C:\Copyfiles\server.txt"
$installerArguments = "/S"  # Silent install argument for Notepad++
$servers = Get-Content -Path $serverList
$logFilePath = "C:\CopyFiles\install_log.txt"

Start-Transcript -Path $logFilePath -Append

foreach ($server in $servers) {
    try {
        $session = New-PSSession -ComputerName $server

        # Copy the installer to the remote server
        Copy-Item -Path $installerLocalPath -Destination $installerRemotePath -ToSession $session -Force
        Write-Host "Installer copy to $server successful"

        # Command to install the software
        $installCommand = {
            param($installerPath, $installerArguments)
            Start-Process -FilePath $installerPath -ArgumentList $installerArguments -Wait
        }

        Invoke-Command -Session $session -ScriptBlock $installCommand -ArgumentList $installerRemotePath, $installerArguments
        Write-Host "Software installation on $server successful"
    }
    catch {
        Write-Host "Error during operation on {$server}: $_"
    }
    finally {
        Remove-PSSession -Session $session
        Write-Host "Session ended for $server"
    }
}

Stop-Transcript

Improvement

1. Can have unit test to validate functionality

2. GUI version where can be user-friendly like can browse the resource file etc.

Let me know if you have any comment.

I helped a friend extract a list of Matta Members (https://www.matta.org.my/members) into an Excel files. With over 2400 entries across 49 web pages, manually copying each entry is impractical and waste of time.

Solution:

Using Python’s BeautifulSoup and Requests modules, I automated the data extraction. If the site required credentials or used JavaScript, I would have needed Selenium or a similar tool.

Result:

Let's get started!

First, we need to know what modules to use.

Python Modules:

import requests
from bs4 import BeautifulSoup
import pandas as pd
import os

BeautifulSoup: Mainly use to extract data from HTML content on webpages. If extract from local, Requests module is enough

How BeautifulSoup work?

1. Loading HTML/XML: The first step is loading the HTML content into BeautifulSoup.

2. Creating the Soup Object: BeautifulSoup parses the content and creates a "soup" object, which represents the document.

3. Navigating and Searching: The soup object allows you to navigate and search through the parsed document easily, using methods like find(), find_all(), select(), etc.

Requests: Make HTTP requests in Python. It will fetch HTML content

Pandas: The library provide data analysis tools. Will use to store and manipulate extracted data in tabular as example below:

OS: The module use to interact with operating system. Will use to handle file path once finish extract and create into csv file

Defining the function

def scrape_matta_members():
    base_url = "https://www.matta.org.my/members?page="
    members = []

def scrape_matta_members(): defines function as per name stated

base_url is the web address for the page that we want to capture. Normally main website is till '?' symbol. After that its on the search or other details.

members = [ ]: It is for initialize empty list to store member details later

Looping

    for page in range(1, 50):
        url = f"{base_url}{page}"
        response = requests.get(url)
        if response.status_code != 200:
            print(f"Failed to load page {page}, status code: {response.status_code}")
            continue

for page in range(1, 50): Will loop from page 1 till 49.

url = f"{base_url}{page}": Constructs the URL for each page by appending the page number to the base URL. Page is int

response = requests.get(url): will send HTTP GET request to the URL that we declare.

if response.status_code != 200: Checks if the request was successful (status code 200 means ok which request succesfully received, understood and accepted)

print(f"Failed to load page {page}, status code: {response.status_code}"): Will print this if unable to capture from the URL

continue: Skips to the next iteration of the loop if the request failed.

Parse

 soup = BeautifulSoup(response.content, 'html.parser')

        # Find the section containing the member details
        member_list = soup.find_all('div', class_='card-box')
        if not member_list:
            print(f"No members found on page {page}")
            continue

soup = BeautifulSoup(response.content, 'html.parser'): Parses the HTML content of the page using BeautifulSoup.

Parses means that it analyze and breakdown the HTML document into structure that easy to navigate. It is converting a string of HTML or XML code into a tree of Python objects

For member_list = soup.find_all('div', class_='card-box') , you need to inspect the website to find the right class.

For this time, all of the details in inside <div class ="card-box"> , hence we will use this to find all div elements that contain all member details.

if not member_list: Checks if no member elements were found.

print(f"No members found on page {page}"): Prints a message if no members were found.

continue: Skips to the next iteration of the loop if no members were found.

Extract and Store Details

        for member in member_list:
            name = member.find('a', class_='search-title').get_text(strip=True)
            reg_number = member.find('span', class_='reg-number').get_text(strip=True)
            contact_number = member.find('span', class_='contact-number').get_text(strip=True)
            web_address = member.find('span', class_='web-address').get_text(strip=True)
            location = member.find('span', class_='location').get_text(separator=", ", strip=True)

            members.append({
                'name': name,
                'reg_number': reg_number,
                'contact_number': contact_number,
                'web_address': web_address,
                'location': location
            })
        print(f"Page {page} scraped successfully.")

Iterates over each member element in member_list and extracts the details.

name = member.find('a', class_='search-title').get_text(strip=True): Extracts the member's name.

reg_number = member.find('span', class_='reg-number').get_text(strip=True): Extracts the registration number.

contact_number = member.find('span', class_='contact-number').get_text(strip=True): Extracts the contact number.

web_address = member.find('span', class_='web-address').get_text(strip=True): Extracts the web address.

location = member.find('span', class_='location').get_text(separator=", ", strip=True): Extracts the location.

Appends the extracted details as a dictionary to the members list.

Convert the extracted data to DataFrame:

# Run the function and get the dataframe
df = scrape_matta_members()

df = pd.DataFrame(members): Converts the list of dictionaries to a Pandas DataFrame.

return df: Returns the DataFrame.

Defining the Save Function

def save_to_csv(df, filename):
    df.to_csv(filename, index=False)
    full_path = os.path.abspath(filename)
    print(f"Data successfully saved to {full_path}")

def save_to_csv(df, filename) defines the function as per mentioned name and takes DataFrame 'df' and filename as argument

df.to_csv(filename, index=False) saves the DataFrame to a CSV file with the specified filename, without including the DataFrame index.

if I put index = True, my csv will look like this:

,index,name,reg_number,contact_number,web_address,location
0,A'Famosa Travel & Tours Sdn. Bhd.,456717-K | MA1999,+60 (06) 5520288,www.afamosa.com,"Club House Building,A'Famosa Resort, Jalan Kemus, Simpang Empat, Alor Gajah, 78000, Melaka, Malaysia"
1,Afbatni Travel & Services Sdn Bhd,1376398-M | MA6643,+60 (03332) 33272,-,"No. 36A, Tingkat 1, Lorong Bayu Tinggi 4c, Taman Bayu Tinggi, Klang, 41200, Selangor, Malaysia"

full_path = os.path.abspath(filename) takes relative path and convert it to absolute path. full_path is variable that will contain the absolute path of file.

print(f"Data successfully saved to {full_path}") is success message

Scrape Data

# Run the function and get the dataframe
df = scrape_matta_members()

df = scrape_matta_members(): Calls the scrape_matta_members function as mentioned earlier and stores the returned DataFrame in the variable df.

Save Data to CSV

# Save the dataframe to a CSV file
csv_filename = 'matta_members.csv'
save_to_csv(df, csv_filename)

• csv_filename = 'matta_members.csv': to give naming for the CSV file.

• save_to_csv(df, csv_filename): Calls the save_to_csv function as defined before to save the DataFrame to the specified CSV file.

Thats all. Comment me if need any other help

full code:

import requests
from bs4 import BeautifulSoup
import pandas as pd
import os


def scrape_matta_members():
    base_url = "https://www.matta.org.my/members?page="
    members = []

    for page in range(1, 50):  # There are 49 pages. It will loop each file
        url = f"{base_url}{page}"
        response = requests.get(url)
        if response.status_code != 200:
            print(f"Failed to load page {page}, status code: {response.status_code}")
            continue

        soup = BeautifulSoup(response.content, 'html.parser')

        # Find the section containing the member details
        member_list = soup.find_all('div', class_='card-box')
        if not member_list:
            print(f"No members found on page {page}")
            continue

        # Iterate over each member entry and extract details
        for member in member_list:
            name = member.find('a', class_='search-title').get_text(strip=True)
            reg_number = member.find('span', class_='reg-number').get_text(strip=True)
            contact_number = member.find('span', class_='contact-number').get_text(strip=True)
            web_address = member.find('span', class_='web-address').get_text(strip=True)
            location = member.find('span', class_='location').get_text(separator=", ", strip=True)

            members.append({
                'name': name,
                'reg_number': reg_number,
                'contact_number': contact_number,
                'web_address': web_address,
                'location': location
            })
        print(f"Page {page} scraped successfully.")

    # Convert to DataFrame
    df = pd.DataFrame(members)
    return df


def save_to_csv(df, filename):
    df.to_csv(filename, index=False)
    full_path = os.path.abspath(filename)
    print(f"Data successfully saved to {full_path}")


# Run the function and get the dataframe
df = scrape_matta_members()

# Save the dataframe to a CSV file
csv_filename = 'matta_members.csv'
save_to_csv(df, csv_filename)

If you read two pages of the Quran a day, you'll finish it twice in a year.

If you walk 10,000 steps every day, it's like running seven marathons in a year.

That's the power of "if".