Lector:

Rubén Lópes
rlopez@brocade.com

Ralph Pawlik, rpawlik@brocade.com
Brocade EMEA

• Frame - 2148 B, 2112B data field (optiona header + payload)
• 5 layers, 0-2

What will be in certificate test:

• 40% around zoning
  • cfgsize
  • merging zones
• routing
• distances
• routing - DLS^IOD - set & reset - high order of delivery
• BCFP - very difficult test
  • ID: 143-5000
  • 60 Q&A
  • 90 minutes
  • well-known adress usage (FW-FD)
• Classes of Service
• long distance vs. trunking
  • what devices will you use in extended Fabric
    • ELWL -Extended Log Way SFP
    • Repeaters
    • WDM

• HW - ASIC, CPU, SDRAM
• ASIC - 90% of switch work
• ASIC - App. Specifig Integrated Curcuit
• Well-known addresses
  • Fabric Login, Fabric Controller, Directory Server
• Brocade doesnt trunk 1Gb port
• Distance:
  1. 1st Level = 100Km
  2. 2nd Level = Bloom ASIC = 100Km; Condor = 500Km (1Gb port)
  3. 3th Level - WAN - pure FC - latency...
• FC router - layer 3
• WWN is in the SDSRAM
• hard zoning using WWN - ASIC go to name server and ask for WWN

• memory - buffer credits - 1Frame big
• Bloom, Bloom II
  • 8x 2Gb ports
  • 108BB - Bloom - 63 on 1 port, Bloom II - 108BB on 1 port
  • ISL Trunking and frame filtering
  • Bloom II - SW3250, SW24000
• Condor
  • 16 x 4Gb per ASIC
  • 8 port group trunking
  • 5% smaller than Bloom
  • buffer memory
    • 1 ASIC = 1024BB
    • 8BB = Hard reserved - cannot be removed
  • can be trunked with Bloom switch
• GoldenEye
  • 24 x 4GB
  • scaled-down Condor
  • 4-port trunk group
  • for smaller platforms
  • SilkWorm 200E

• Brocade doesnt trunk 1Gb port
• follow quad group
• speed same everywhere

• ports from right to left
• port can be preconfigured
• no demo licence, only fow SW - fabric manager
• adding by root

• hashow - look to synchronization status

• Fabric Watch - switch only tool; monitoring and collection tool for ports
  • could set boundaris for monitor (for example more then 80Mb, every minute)
  • 2 triggers - continual trigger, or only once
  • above, bellow, in-between, exceed, change
  • snmp
  • rapiTrap, PortLogLock (stop collecting), E-mail
• Advance Performance Monitor APM
  • protocol - IP/SCSI general
  • source/destination ID
  • LUN association - between initiator-target
• Ports on Demand
• FCIP
• Advanced Zoning
• Web Tools
• ISL Trunking
• Secure Fabric OS
• Extended Fabric
• Standard Features
  • Radius, SNMPv3. SSH, SSL, FICON, SMI-S

• every switch has embeded port ID - switch ID

• PLOGI - port login - name server
• FLOGI - fabric login - fabric login
• SCR - state change registration - Fabric controller
• QUERY - initiators sent (targets listen)
• RSCN - Remote state change notifications (sents onlyto initiators)

1. local
2. remote

• database contains
  • Port attributes
    • port identifier (native portaddress)
    • Port name (port WWN)
    • Cass of Service (2,3)
    • FC-4 types (FCP=SCSI,FC-IP...IP in FC)
    • Port Type (N,L)
    • Symbolic Port Name
  • Node attributes
    • Node name (NWWN)
    • Fibre channel IP adress
    • Symbolic Node Name (free-form initiator)

• nsshow - local database in the switch
• nscamshow - remote name server database
• nsallshow - display the 24bit address of all devices in the fabric

• NTP protocol
• one principal switch in fabric is responsible for sending NTP

• responsible for managing fabric - single point

• 1-4Gb
• media

1. kodování a dekoding
   • z 8b na 10b (64b/66b for 10Gb propriterary)
   • slovo = 4znaky
     • znak K - spec znaky a D - data
2. detekování chyb - CRC
   • only detect, never drop frame because of IoD - In order of Delivery
     • target decide

Frames

• firmware level
• class od service
  • mechanizmus pro přenos dat

1. Class 1 - potvrzované vyhrazované spojení ACK - pro streaming
2. Class 2 - mezi porty, switch je zodpovědný za doručení, ACK
3. Class 3 - jako Class 2, ale nepotvrzovaný
4. Class 4 - jako Class 1, vytváří virtual circuits pro více spojení najednou. garance pořadí doručení
5. Class F - switch-to-switch

Potvrzení přijmutí:

• end-to-end
• buffer-to-buffer - v bufferech karet

• E-port - 26BB by default
• pathinfo - count of BB on switch

Distributed fabric services:

1. name server - wwwn
2. time server
3. alias server - alias na wwn

• NAT

Definuje jak pracovat s protokoly - upper level protocol:

• SCSI
• IP
• virtual intervace (VI)
• Intelligent peripheral interface (IPI)
• High Performance Parallerl Interace (HIPPI) Framing Protocol
• ...

Map in the payload.

• each switch is responsible for assigning unique addresses
• addresses are 24b
  1. Domaind ID 8b - 0×01-0xED
  2. Area ID 8b - 00×00-0xFF
  3. Node Adress 8b - 00 or AL_PA
• adress types
  • Fabric: DD AA 00
  • Public loop: DD AA PP
  • Private loop 00 00 PP

• users: 4xroot, 4xfactory, 4xuser, 1xconsole, 2xadmin =15xconnection
• defalt accont: admin - FAOS
• default password: password
• other accounts: root - Linux ,factory - Linux , switchadmin - FabricOS - junior admin and user - FOS read only

• killtelnet
• quetmode - usefull for setup
• bannerset “anything” - switch centric -cnnot be propagated
• configure - basic config, (RPC for Fabric Manager)
  • set Domanin ID always
• configdefault - all default (not set WWN, MAC, IP, SNMP, Zoning, Switch name, Licence)
• configshow fabric.ops - switch centric
• fabricshow - verify fabric member
• switchdisable(enable)
• timeot 15 - CMD line timeout
• uptime
• switchstatusshow
• switchstatuspolicyshow(set) - when LED will change

• hareboot - not destruktive, fastboot, reboot ~10min., switch reboot
• hashow - HA synchronize!!!

• firmware(download|restore) -s
• firmwareshow
• version - running firmware Fabric OS on Central Procesor

• portname slot/port(port)
• portdisable(enable) slot/port
• portsfgpersistentdisable(enable) slot/port
• portshow port

• portcfgspeed port,speed_level
• switchcfgspeed speed
• switchshow - if Mod_Inv than port speed doent matche
• portcfgshow - speed, trunking and port type
• portcfgdefault
• portcfg(e|l|g)port

• passwdcfg –set ... - password policies
• passwdcfg –showall
• passwd - password change
• passwdcfg –setdefault
• passwd –showall
• userconfig –show(delete|add|change) - administrate accounts

• syslogipadd
• syslogipremove
• syslogipshow

• switchname “...”
• fabricshow

Case sensitive

• licenseidshow - WWN ID switch for licencing
• licenseshow - show licence
• licenseadd - add
• licenseremove

• tsclockserver timeserver-IP(LOCL) - timeserver NTP - will be propagated to other switches
• date
• timezone -5 - timezone config

• ifmodeshow eth0
• ifmodeset eth0 - speed seting
• ipaddrset - IP address seting

• psshow
• fanshow
• tempshow
• sensorshow

https://remotesanlab.brocade.com
user: rsl1_st14
passwd: ruben2020
Ctl+Pause

windows

rsl1_st14
ruben2020

firmware

/u5.x/u5.1.0/release.plist ... symbolic list depended on switch type
IP: 10.255.252.50
  - uses ftp protocol

Zoning is using logical bariers on physical devices. Could be on or off.

• port zones - before Bloom
• WWN zones - WWN are in name server distributed; more flexible
• zones could overlap - devices could be in more zones
• every switch need a licence
• switch is allways using effective configuration
• changes must be changes → but they are not effective
• after cfgenable will be config effective
• RAM size - old switches often must be separated because of less Memory
• If zoning is enabled and device is not defined in enabled zone configuration, it doesn’t exist.

Policy based administration:

• defined vs. effective configuration - cfgshow

• do not disable zone if you need to made new changes to effect
• NEVER issue cfgdisable on productive HW - all switches in fabric will be cleared
• backup configuration to ftp - configupload, Fabric Manager is better
• develop naming convention - easily identify devices
• identify members by port or WWN
• create aliases, zones, zone configuration
• exclude E_ports, that are shared devices
• monitor zone database size
• analyze zone configuration - nszonemember,fcping, SW tools
• enable the zone configuration
• verify there is accessibility between zone members

1. Make sure switch is disconnected!!!! (E_ports too)
2. cfgshow’cfgdisable; cfgclear;cfgsave
3. Connect to existing fabric
4. Defined and effictive configurations are propagated to new switch

• CLI
• Web Tools
• Fabric Manager - good for connection check
• SAN Health - it’s free; very usefull for checking zones

• case sensitive
• up to 64 characters, be aware of memory
• members
  • <domain, port> or L<domain,area>
  • NWWN - nsshow
  • PWWN - portloginshow
• sample
  • SRV - server - SRV_SunHost
  • STO - storage
  • TPA - Tape

• everything must be in the same order
• Check by Fabric manager before connect
• or Segmentations:
  1. Configuration mismatch
  2. Type mismatch
  3. Content mismatch

• cfgsave & cfgshow
• cfgenable - enable zones config and save it
• cfgdisable, cfgclear - DO NOT USE!!! on working HW
• cfgtrans(show|abort) - displays current transaction

alicreate "Eng_Host","1,0"
...
cfgsave
zonecreate "Zone_Eng","Eng_Host; Eng_Stor"
...
cfgsave
cfgcreate "Cfg_EngMkt","Zone_Eng; Zone_Mkt"
cfgshow
cfgsave
cfgenable "Cfg_EngMkt"
switchshow

Fabric OS v5.1.0:

• defzone –(all|no)access - set the access mode - default zone; prevent cfgdisable
• cfgsave & cfgenable
• defzone –show

(not in the test)

In the Fabric in old Fabric OS 5.0.1 switches it looks as normal zone, but not have a commands for administration.

• cfgshow
• cfgdisable, cfgclear, cfgsave
• portzoneshow - what level of access control is aplied
  1. Hard Port - All Domain/Port
  2. Hard WWN - All WWN, zone is defined with WWN
  3. Session Hard - mixed zone; no CAM entries at
• cfgactshow - display the effective configuration
• portcamshow - show how much CAM memory is used; could by a issue
  • 2G Bloom - 64sid/512/did - Quad
  • 4G Condor - 2048sid/1024did - per 32ports
• cfgsize - how much memory is used/left
• nsaliasshow
• nodefind Alias|wwn|PID -find device
• nszonemember 0x04006e2 - which device is communication inside zone
• nszonemember -u display all unzoned devices

• zoneobjectcopy - copies a zone object to a new zone object
• zoneobjectrebane
• zoneobjectexpunge - deletes the zone object and remove it from the member list of all other objects

• Path=LSD-Hop/Weight; - in SDRAM
• Route=ASIC - path with minimum cost
• do not change domanin IDs
• routing is set to the target ⇒ link cost could be asymetrical depend on the way
• Principal Switch
  • responsibility for initialize The Fabric, before routing is established
  • manages the assignment of unique domain IDs
  • provides time sync of all switches in the fabric
  • fabricshow → fabricprincipal [-f] 1 - set the preffed Principal switch
• Principal ISL (inter switch link)
  • ISL used to communicate between the Principal Switch
• Routing policies:
  • 2Gb - port-based routing
  • 4Gb - port and exchange-based routing
• DLS(set|reset) - when link goes down - Dynamic Load Share set/reset; - devices
• IOD(set|reset) - E_ports
  • reset=150ms
  • Hold_time=150ms = (e_d_Tov-R_A_TOV/1)
• high order of delivery:
  1. DLSset
  2. IOD set

• 8 VC queues are used per ISL
• used to solve traffic congestion by interleaving frames

^ Priority ^ Channel ^ BB credits^

• by Brocade
• calculates minimum cost path from switch-to-switch
• downloads route tables to the ASIC
• cost:
  • 1000 - 1Gb/sec
  • 500 - 2|4GBit/sec

• fabricshow
• fabricprincipal [-f] 1 - set the preffered principal switch
• lincost - change cost of link; must be executed in every hop (0~disable)
• urouteconfig - configures a static route
• switchdisable; aptpolicy 1|2|3 - 4Gb only (do not do that)
• topologyshow - how one domain is communicating with other domain
• pathinfo -r 1 - routing and statistic information between a source and destination ports

• reuiments
  1. licence
  2. speed
  3. quater port groups
• Round-robin
• mantains in-order delivery
• diferences between 2 and 4Gb
  • in Condor uses link in trunk only when it’s needed (8GB → 2 links from8, 10Gb → 3 links)
• cable lenght between shortest ISL and Logest ISL in trunk
  • max cable difference - 400m
  • differences > than 30m could introduce performance degradation
    • 30m ~ 15 deskew (trunkshow)
    • shortest path ~ 15 deskew
    • above 30 deskew indicates degradation

• Can not be set or removed by configure
• can be cleared by portcfgdefault
• Saved in switch configuration file configupload as portcfg parameter
• all port in trunk grup must set to the same L-mode
• BB credits
  1. Bloom 108bb per 4-port group (63 on single port)
  2. Condor - max 100bb per 32 ports, 255BB to each one
  3. GoldenEyes - up to 288 for all 16 pots, maximum 128bb to any one port
• Modes:
  1. LE - Bloom
  2. L0.5 - 0.5Km
  3. L1 - static mode 50Km 1-4Gb
  4. L2 - 100Km
  5. LD - Dynamic mode uses automatic detection and a user defined distance
  6. LS - static mode that allows a user-defined diistances - Fabric OS above v5.1.

• portcfgshow
• switchcfgtrunk 0|1 - will turn trunking on all ports
• trunkshow
• islshow
• portcfgtrunkport
• switchcfgtrunk
• trunkdebug <start port <end port>
• portcfglongdistance 3 LS 1 75 - 1~VC link init; 3~port; 75km LS distace level
• portbuffershow - show buffer allocation
• portcfgdefault

trackchangeset 0

#clear buffer log
errclear

#print log
errdump

defzone --noaccess
cfgsave

#dump to ftp file
supportftp -s -h host-IP -u user-ftp -p password -d remote-directory
traceftp -e 
supportsave
#print debug
supportshow

#test
porttest
#print port test
porttestshow

• portcfgshow - display the setting fow all ports on a switch
• fabricshow
• switchshow
• help, help command

• aaaconfig –show - RADIUS configuration show
• aaaconfig –add IP -s SECret123
• aaacongig –radius on - enable using radius
• aaaconfig –switchdb on - enable using local user db
• userconfig –show -a - do not show Radius users

• RTOS - Realtime OS - Linux; initial at boot, provide SSH, snmpd
• FOS - Fabric OS - as Linux apps; provide Fabric Servises
• APNM - advance performance monitor; collect information about port - fabric manager used it
• WWN - 64b unique world wide name; for binding
• CAM - content addressable memory; WWN + 24b source addr.
• HAM - HA monitor - in directors - IP heard beat
• FSS - fabric sync. service - in directory synchronization monitoring; know state od ASIC all the time
• LC - Lucent Connector
• SC - siemens Connector
• SFP types
  • SWL - short Wavelenght
  • LWL - long Wavelenght
  • ELWL - extended Long Wavelenght
  • WDM - wave division multiplexing
• FCIP - FC over IP
• IoD - In order Delivery !!!!
• UPC - Upper Level Protocol - FC 0
• NWWN - node WWN - client (20:00:
• PWWN - Port WWN (10:00:, 21:00:,22:00:)
• LIP - Loop Initialization Primitive
• BF - build fabric - Hold_Time
• Trunk(back) Master - responsilble for building trunk
• FSPF - Fabric Shortest Path First (like OSPF)
• ISL - inter switch links
• DLS(set|reset) - when link goes down - Dynamic Load Share set/reset; - devices
• IOD(set|reset) - E_ports
  • reset=150ms
  • Hold_time=150ms = (e_d_Tov-R_A_TOV/(max-hops

• FC docs
• Certification